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Executive Summary 


This study examined outcomes for AP® Calculus and AP Physics students on the 2015 
Trends in International Math and Science Study (TIMSS) relative to other participating 
countries and other advanced math and science students in the United States. Compared to 
other countries in the study, AP Calculus BC and AP Physics C: EM students outperformed 
all participating education systems and other AP groups. Data also show that higher 
percentages of AP Calculus and Physics students reached international benchmarks, 
earned higher average content domain scores, and spent more time engaged in math and 
science than other advanced math and science students in the U.S. who did not take AP. 
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Introduction 


Education scholars from several nations met in Hamburg, Germany, in June 1959 to discuss 
“an international study of intellectual functioning.”' No scientifically rigorous international 
assessment had ever been attempted before. Many doubted that student learning could be 
assessed accurately across countries with vastly different languages, cultures, and 
educational systems. The group of researchers, which was later formally named the 
International Association for the Evaluation of Educational Achievement (IEA), planned a 
pilot study to determine whether an international project was even feasible. The pilot study 
was completed in 1960. It convinced the IEA’s founders, who included Benjamin Bloom of 
the University of Chicago and Robert L. Thorndike of Columbia University, that an 
international assessment was not only feasible, but could also produce meaningful 
comparisons among participating nations. 


Math was selected as the first school subject for assessment. The first international math 
study (FIMS) was conducted in 1964 in 12 countries. The final sample consisted of 132,775 
students and 13,364 teachers from 5,348 schools. Students were assessed at two points in 
schooling—age 13 and the final year of secondary school (i.e., senior year in American high 
schools). This being a time of limited computing and communication facilities, it took three 
years to collect, to process, and to analyze the data (Trosten 1967). 


The results were released in a two-volume report published in 1967 (Husen 1967). The 
United States’ performance was disappointing. For 13-year-olds, the U.S. ranked next to 
last, in 11th place. American researchers had already anticipated low scores. The 
Washington Post headline of March 12, 1967 read, “Poor U.S. Math Showing Didn't 
Surprise Experts,” noting, “The experts assert that teachers here are not as well trained, and 
that neither American students nor the society at large places as much value on 
mathematics achievement as do many countries abroad.” The New York Times bluntly 
declared, “The U.S. Gets Low Marks in Math.” 


FIMS was followed by the Second International Math Study (SIMS) in 1980-81 and the Third 
International Math and Science Study (TIMSS) in 1995. Having decided to conduct TIMSS 
every four years, the project later kept the TIMSS acronym but changed what it stood for, 
becoming the Trends in International Mathematics and Science Study. TIMSS 1995 focused 
on students in two elementary school grades (third and fourth) and two in middle school 
(seventh and eighth grades). TIMSS was repeated in eighth grade in 1999 (called TIMSS- 
R). Beginning in 2003, all subsequent TIMSS assessments were administered in fourth and 
eighth grades. 


Another 1995 innovation was the addition of TIMSS Advanced, an assessment targeting 


students in their final year of secondary school who are taking advanced math and science 


1. This idea was expressed in a 1958 proposal to the UNESCO Institute for Education. 
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courses. Nineteen countries took part. TIMSS Advanced was administered again in 2008 
and 2015, with 10 countries participating in 2008 and 9 countries in 2015 (see Table 1). 


Table 1: TIMSS Advanced Participating Countries 


e = Indicates participation in particular assessment with results reported or forthcoming. 
1 Administered physics but not advanced mathematics in 1995. 


2 Administered advanced mathematics but not physics in 1995. 


Education System 
Armenia 

Australia 

Austria 

Canada 

Cyprus 

Czech Republic 
Denmark 

France 

Germany 

Greece 

Iran, Islamic Republic of 
Israel 

Italy 

Latvia! 

Lebanon 
Lithuania? 
Netherlands 
Norway’ 
Philippines 
Portugal 

Russian Federation 
Slovenia 

Sweden 
Switzerland 


United States 


1995 


Last-year Secondary _Last-year Secondary 


School 


Note: OECD member countries are bolded. 


o 


2008 


School 


2015 


Last-year Secondary 
School 
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Source: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics 
and Science Study (TIMSS); this comes from the website, https://nces.ed.gov/timss/countries_advanced.asp 


TIMSS Advanced 2015 


The 2015 TIMSS Advanced test assessed student knowledge in advanced mathematics and 
physics. In terms of content, the physics test covers mechanics and thermodynamics, 
electricity and magnetism, and wave phenomena and atomic/nuclear physics. The 
advanced mathematics assessment covers algebra, geometry, and calculus. In both 
subjects, three cognitive domains (knowing, applying, and reasoning) specify thinking 
processes that students employ to address the items. 


Each participating country determines student eligibility for TIMSS Advanced. In general, 
students must be in the final year of secondary education and enrolled in the math or 
physics course serving each nation’s most advanced students, typically those intending to 
study mathematics or science in college. An individual student may belong to the advanced 
math target population, the physics target population, or both. U.S. students who belong to 
both are randomly assigned either to the math or physics assessment, ensuring that each 
student participating on TIMSS Advanced only takes one test. Matrix sampling is used for 
both assessments, with six booklets in advanced math and six booklets in physics. Each 
student completes a single booklet. Booklets are distributed among the students in each 
sampled class according to a predetermined order, so that approximately equal proportions 
of students respond to each booklet. 


Scores are placed on achievement scales so that changes in achievement can be estimated 
over time. The TIMSS Advanced scales are based on the 1995 assessments, with the scale 
centerpoint set at 500, equal to the international average across all participating countries in 
1995, and one standard deviation equal to 100 scale points. Including common items on 
1995, 2008, and 2015 administrations of TIMSS Advanced has allowed the linking of scores 
over the entire 20-year period so that countries can calculate progress in advanced math 
and physics performance (more on TIMSS methods can be found in this report's 
methodology section). 


In addition to the assessments, TIMSS Advanced collects data through questionnaires 
distributed to students, teachers, and school principals. The questionnaires gather 
information on factors that may affect student learning. These factors include: 


e Students’ academic preparation for advanced math and physics 
e Students’ educational aspirations 
e Students’ attitudes toward math and science 


e Teachers’ education and training 
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e Teachers’ instructional strategies 


e Use of technology by students and teachers 


Our Study 


The questionnaire topics listed above are interesting because they may be related to 
student achievement. But that does not mean TIMSS data allow for rigorous tests of 
causality. Students are not randomly assigned to advanced math and physics courses. 
Moreover, even though the selected samples are designed to be representative of the 
advanced students from which they are drawn, cross-sectional data only allow for estimates 
of phenomena taking place at a single point in time. Thus, the study below is primarily 
descriptive. We seek to explore how advanced students in the U.S. performed on the TIMSS 
Advanced test in 2015 and by closely examining questionnaire data from students, teachers, 
and school administrators, to describe the contexts in which teaching and learning occurred. 
These are the nation’s most advanced students in math and science. We hope that our 
report allows readers to know them a little better. 


Our study contains an element that previous studies of U.S. performance on TIMSS 
Advanced have not been able to pursue, the ability to disaggregate data by Advanced 
Placement® (AP) Program status. The AP Program provides students the opportunity to take 
college-level course work while in high school. AP offers two courses in calculus—Calculus 
AB and Calculus BC—and four courses in physics—Physics 1, Physics 2, Physics C: 
Electricity and Magnetism, and Physics C: Mechanics. College and university faculty review 
AP courses to ensure alignment with college-level expectations (full course descriptions can 
be found online). AP courses conclude with a culminating exam. Many colleges and 
universities accept a successful score (typically 3 or above on a 5-point scale) for credit or 
advanced placement.® 


The study is guided by two key questions: 


1. How do AP and non-AP students compare, both to each other and to students 
internationally, in performance on TIMSS Advanced? This question is tackled in section 
3: 


2. What are some of the contextual factors that differentiate the learning experience of AP 
and non-AP students? The analysis is presented in section 4. 


We organize the contextual factors by themes: student preparation, technology, teachers 
and teaching, and student attitudes and aspirations. 


2. https://apcentral.collegeboard.org/courses 


3. AP Program Guide 2016-17 (College Board, 2016). 
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Student Preparation 


Education reform movements spurred by Sputnik in the 1950s and A Nation at Risk in the 
1980s urged American students to take more challenging courses, particularly in math and 
science. Several studies have documented the correlation of advanced course work in high 
school with high achievement and later success in college.* Leow et al. (2004) used 
propensity scoring to investigate the effect of advanced course taking on performance on 
the TIMSS 1995 assessment. The authors found that taking advanced math and science 
courses significantly associated with higher achievement compared to non-advanced course 
taking. 


Less research has been conducted on how students are prepared for taking advanced math 
and science in high school and the specific pathways advanced students travel while 
studying the two subjects. Advanced science has historically comprised courses in biology, 
chemistry, and physics, but local customs and policies dictate the sequence of offerings. 
Although the math sequence has always been more clearly defined (Algebra |, Geometry, 
Algebra II, and Calculus), enrollment in the first steps of the continuum have recently been 
in flux. Enrollment in advanced math courses (Algebra | or higher) in eighth grade nearly 
tripled from 1990 to 2011, going from 16% to 47%.° Unfortunately, recent studies of student 
transcripts reveal an extremely leaky pipeline. A California study, for example, found that 
57% of students took Algebra | for the first time in eighth grade, but three years later only 
15.3% had passed Algebra |, Geometry, and Algebra II in grades 8, 9, and 10, respectively.® 


TIMSS Advanced students were asked to report the grade in which they took math and 
science courses, beginning with eighth grade. We analyze those responses to find out who 
took which courses and when they took them, mapping the curricular pathways students 
traversed to arrive, finally, as high school seniors, in advanced calculus and physics 
classrooms. 


Technology 


Advocates argue that greater use of technology will improve curriculum and instruction, 
leading to increased student achievement and enthusiasm for academic subjects. Recent 
reports indicate that many schools are not able to make use of the benefits of technology 
due to a lack of resources, expertise, and access to reliable information (NEA 2008). Lytle 
(2011) reports that K-12 teachers who implemented technology in their classrooms believe 
that it positively affected their students’ achievement and productivity. While schools and 


4. Adelman (1999). Answers in the Toolbox: Academic Intensity, Attendance Patterns, and Bachelor’s Degree Attainment; 
Leow et al. (2004); Long, Conger, and Lataola (2012). Effects of high school course-taking on Secondary and 
PostSecondary Success, American Educational Research Journal, 49 (2), pp. 285-322. 


5. NAEP data, as reported in Loveless (2013), “Advanced Math in Eighth Grade,” 2013 Brown Center Report on American 
Education. 


6. Finkelstein et al. (2012), “College Bound in Middle School and High School?” WestEd. 
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districts have begun infusing technology into the curriculum, about 27% (according to a 
study by CompTIA 2011) of educators still experience budgetary constraints and other 
obstacles that hinder efforts. 


Technology permeates every aspect of students’ lives, perhaps to a larger extent outside of 
school than within. We examine data from the TIMSS Advanced questionnaires for both 
teachers and students to explore the role of technology in the study of math and science. 


Teachers and Teaching 


Shortages in the pool of qualified STEM teachers have been a policy concern for a long time 
in the U.S. Research has shown that teacher experience is positively correlated with student 
achievement, but the relationship weakens after five years of experience teaching.’ A policy 
concern that arose in Norway after the release of 2008 TIMSS Advanced was the 
percentage of teachers who were approaching retirement age.® All over the world, advanced 
math and science courses tend to be staffed by schools’ most experienced teachers, so 
their retirements represent a large loss of human capital. 


TIMSS questionnaires ask teachers about their age, level of education, and teaching 
experience. They also ask about instructional strategies and confidence in the ability to 
engage in several teaching practices. Because pedagogical reform has been the focus of 
math and science policy efforts in recent years, an analysis of teachers’ responses will be 
timely. 


Student Attitudes and Aspirations 


Adolescents’ aspirations and attitudes influence school achievement. A longitudinal study by 
Beal and Crocket (2010) on 317 adolescents investigated future aspirations, current 
activities, and educational attainment (see also, Gregory and Weinstein 2004). Regression 
analyses revealed that adolescent career aspirations (“What kind of work would you like to 
do?”) predicted later educational attainment. Adolescents with professional work aspirations 
were more likely to attain higher levels of education later in life. 


Several studies also show that students who like math and science tend to do well in them.? 
It makes sense that students who enjoy particular school subjects and envision them as part 
of their future careers would also be high achievers in those subjects. That said, the 
direction of causality, which comes first, attitudes or achievement, is ambiguous. Two 


7. Rivkin, et al., 2001. Also see Jonah E. Rockoff (2004), “The Impact of Teachers on Student Achievement: Evidence from 
Panel Data” American Economics Association Papers and Proceedings, May 2004, 247-252. 


8. Gronmo, L.S., Onstad, T., Pedersen, |.F., Lie, S., Angell, C., & Rohatgi, A. (2009). Mathematics and physics in upper 
secondary school. One step back. Abridged report presenting main results from TIMSS Advanced 2008 in Norway, 
Department of Teacher Education and School Development, University of Oslo. 


9. Kusum Singh, Monique Granville, Sandra Dika (2002). “Mathematics and science Achievement: Effects of Motivation, 
Interest, and Academic Engagement, The Journal of Educational Research, vol. 95, no. 6, 323-332. 
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patterns from international assessments add to the ambiguity.'° On both TIMSS and the 
Programme for International Student Assessment (PISA), the relationship between attitudes 
and achievement at the national level tends to be negative; that is, higher scoring countries 
in math and science have higher percentages of students saying they don’t like the 
subjects—and vice versa. Students in lower scoring countries regard math and science 
much more positively. 


Another observed pattern is that attitudes toward subjects are negatively correlated with 
age. Older K-12 students view subjects more negatively than younger students. The 
students in TIMSS Advanced are both high achievers and in the final year of high school. 
We examine responses to student questionnaires that reveal how America’s best students 
in mathematics and science view both subjects and their educational and career aspirations 
for the future. 


Methodology 


Introduction 


In this section, we present the methodology used to estimate achievement on TIMSS 
Advanced for particular subgroups and the analysis of the questionnaire data. These 
subgroups are specifically defined in this report to identify students who have taken an AP 
Exam and students who havenot participated in an AP Calculus or AP Physics course. This 
section will also outline the sampling design used in the U.S. for TIMSS Advanced 2015 and 
explain how the TIMSS coverage index is calculated for each participating country. 


Sample Design for the U.S. in TIMSS Advanced 2015 


The purpose of the TIMSS Advanced assessment is, “to provide valid and reliable 
measurement of trends in student achievement in countries around the world.” \n the United 
States, a nationally representative target sample was designed with input from the National 
Research Coordinator, the College Board, sampling staff from the IEA Hamburg, and 
Statistics Canada. This sample explicitly included students in AP Physics and AP Calculus 
courses, as well as students in other advanced math and physics programs (e.g., 
International Baccalaureate, dual-enrollment, etc.). In addition, the sampling frame was also 
expected to, “identify the programs, tracks, or courses that correspond to the international 
target population; create a sampling frame by listing all schools in the population that have 
classes with advanced mathematics and/or physics students in the target grade; determine 
national population coverage and exclusions, in accordance with the TIMSS Advanced 
international guidelines; work with Statistics Canada to develop a national sampling plan 
and identify suitable stratification variables, ensuring that these variables are present and 
correct for all schools; contact all sampled schools and secure their participation; keep track 


10. Ina Mullis, Michael Martin, and Tom Loveless, (2016). 20 Years of Trends on TIMSS. 
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of school participation and the use of replacement schools; and conduct all within-school 
sampling of classes” (LaRoche and Foy 2016, 2). It must be noted that the U.S. did not meet 
the sampling requirements on TIMSS Advanced 2015. A complete outline of the TIMSS 
Advanced international guidelines for sampling can be found in the Methods and 
Procedures in TIMSS Advanced 2015 (2016). 


Courses Included in Advanced Mathematics and Physics 


While the TIMSS Advanced 2015 Assessment Framework outlines the content for the 
courses that define the target population of students, each country chose the courses to be 
included in the sample that they felt most closely matched this content. Often, these were 
the most advanced mathematics and physics courses each country had to offer. For 
mathematics in United States, the courses included, “students in the 12th grade who have 
taken an advanced mathematics course (AP, IB, or another advanced mathematics course 
specific to their state/district), in grade 12 or in a prior grade” (LaRoche and Foy 2016, 4). 
For physics in the United States, the courses included, “students in the 12th grade who have 
taken an advanced [physics] course (AP, IB, or another advanced physics course specific to 
their state/district), in grade 12 or in a prior grade” (LaRoche and Foy 2016, 4). For more 
details on U.S. sampling, see https://nces.ed.gov/timss/timss15technotes_sampling.asp. 


TIMSS Advanced Coverage Indices 


The TIMSS Advanced coverage index for math and physics is a measure of the “school- 
leaving age cohort taking advanced mathematics and physics courses” in each country 
(LaRoche and Foy 2016, 5). These indexes identify the “overall sampling coverage” for each 
country’s population and represent the total percentage of the corresponding age cohort that 
would be considered “eligible” for TIMSS Advanced 2015 in each country. As outlined in the 
Methods and Procedures in TIMSS Advanced 2015, the TIMSS Advanced coverage 
indexes are calculated as follows: 


Estimated total number of students in 
the advanced mathematics population 
Total national population in 
the corresponding age cohort 


TIMSS Advanced Math Coverage Index = x 100% 


Estimated total number of students in 
the advanced Physics population 
Total national population in 
the corresponding age cohort 


TIMSS Advanced Physics Coverage Index = x 100% 


Table 2 shows the calculated coverage indexes for the United States in both 
Advanced Mathematics and Advanced Physics. 
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Table 2: U.S. Coverage Index 
a a a a 
Estimated Size Size of the Age TIMSS Advanced 


of the Cohort Coverage Indices 
Population of Corresponding — the Percentage 
Age Cohort Students in to the TIMSS of the Entire 
Corresponding _ the Final Year Advanced Corresponding 
to the Final of Secondary Population Age Cohort 
Years of Year of School Taking Based on Covered by TIMSS 
United Formal Secondary Advanced National Census Advanced Target 
States Schooling School Math/Physics Figures Population 
Math 12 18 473,405 4,168,000 11.4% 
Physics 12 18 199,944 4,168,000 4.8% 


Source: Adapted from LaRoche & Foy, 2016, p.6 & 7. 


Operationally Defining the Advanced Placement Groups 


For the purposes of this report, AP student groups were defined using data obtained from 
College Board about students who took specific AP Exams during 2015 and TIMSS 
Advanced 2015 data from the National Center for Educational Statistics. For TIMSS 
Advanced Math, we identified U.S. students who participated in TIMSS Advanced Math and 
who also took the AP Calculus AB Exam or the AP Calculus BC Exam (no students took 
both exams). These students were categorized as “AP students.” All other TIMSS Advanced 
students were categorized as “non-AP students.” 


In analyzing the group of non-AP students, we found that 697 of these students who did not 
take an AP Exam in 2015"! self-reported that they had taken or were currently enrolled in an 
AP Calculus AB course, and that 230 of these students self-reported that they had taken or 
were currently enrolled in an AP Calculus BC course (126 students self-reported they took 
both AB and BC courses). These students were deleted from the non-AP group. Thus, the 
final groups for this report’s analyses were defined as: 


e AP Calculus AB Students: Students who took the AP Calculus AB Exam in 2015 who 
also took the TIMSS Advanced Mathematics assessment in 2015 


e AP Calculus BC Students: Students who took the AP Calculus BC Exam in 2015 who 
also took the TIMSS Advanced Mathematics assessment in 2015 


e Non-AP Students: Students who did not take the AP Calculus AB or BC Exams in 2015, 
AND who did not self-report ever taking either of the Calculus AB or BC courses, who 
also took the TIMSS Advanced Math exam in 2015 


11. There were initially 1,454 students in the non-AP Calculus group. 
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For TIMSS Advanced Physics, we identified U.S. students who participated in TIMSS 
Advanced Physics and who also took the AP Physics 1, AP Physics 2, AP Physics C: 
Electricity and Magnetism (AP Physics C:EM), or the AP Physics C Mechanics (AP Physics 
C:M) exams. We initially created separate AP groups based on the AP Exams taken, but 
there were a small number of students who took more than one of these exams. Therefore, 
we decided to form the AP groups based on the “highest” AP Physics Exam taken, with 
Physics 1 being the lowest in the hierarchy, and Physics C:EM being the highest. Given that 
the Physics 2 group was only 55 students, we omitted them from the analysis. Students who 
met these criteria were categorized as “AP students.” All other TIMSS Advanced students 
were categorized as “non-AP students.” 


Similar to the TIMSS Advanced 2015 Advanced Mathematics data, there were many 
students in the TIMSS Advanced Physics non-AP group who responded they had taken or 
were currently enrolled in an AP Physics course (954 students"). These students were 
omitted from the non-AP group. Thus, the final groups were defined as: 


e AP Physics 1 Students: Students who took the AP Physics 1 Exam (or students who 
took both AP Physics 1 and Physics 2 Exams) in 2015 who also took the TIMSS 
Advanced Physics exam in 2015 


e AP Physics C:M Students: Students who took the AP Physics C:M Exam in 2015 who 
did not take the AP Physics C:EM Exam, and who also took the TIMSS Advanced 
Physics exam in 2015 


e AP Physics C:EM Students: Students who took the AP Physics C:EM Exam in 2015, 
who also took the TIMSS Advanced Physics exam in 2015 


e Non-AP Students: Students who didn’t take an AP Physics Exam in 2015, AND who 
didn’t self-report ever taking a physics course, who also took the TIMSS Advanced 
Physics exam in 2015 


In Table 3 we report the sample sizes for these groups for physics and mathematics. It 
should be noted that for the analyses reported in the body of this report, we applied the 
TIMSS Advanced student house weights, so that the results would reflect population 
inferences, rather than just characteristics of the samples. 


' There were initially 1,692 students in the non-AP Physics group. 
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Table 3: Calculus and Physics: Analytical Group Sample Sizes 
a Se ee SSS SaaS 


Group N Percentage 

AP Calculus AB 1,151 52.3 

AP Calculus BC 398 18.1 

Non-AP 653 29.7 
Total 2,202 100.0 

AP Physics 1 713 36.0 

AP Physics C:M 397 20.0 

AP Physics C:EM 133 6.7 

Non-AP 738 37.3 
Total 1,981 100.0 


Note: Weighted, after exclusions. 
Analysis of Questionnaire Data 


TIMSS Advanced included questionnaires for students, teachers, and schools. The teacher 
and school questionnaire data were merged with the student questionnaire data, so that the 
results reflected the weighted proportions of students, rather than the numbers of teachers 
or schools responding. 


Note to Readers About Questionnaire Data 


The TIMSS Advanced sampling design produces a nationally representative sample of 
students in the target population. As a consequence, data from questionnaires given to 
teachers and school administrators are expressed in student units. In the report’s discussion 
of teacher and school characteristics, for example, readers may encounter, “More than 80% 
of students were taught by a teacher with the following characteristic,” or “More than 80% of 
students attended a school with the following characteristic,” rather than seeing results 
reported as “percentage of teachers” or “percentage of schools.” This convention sometimes 
leads to awkward language, but more precisely conveys the inferences that can be drawn 
from the study’s data. 


Estimating TIMSS Advanced Achievement Using Plausible Values 


The TIMSS Advanced program publishes plausible values (e.g., von Davier, Gonzalez, and 
Mislevy 2009) as part of its statistical analysis for studying student achievement. In addition, 
plausible values are published for the convenience of secondary data-analysts, as the 
plausible values facilitate appropriate statistical inference when modeling student 
achievement on TIMSS Advanced. TIMSS Advanced also uses stratified sampling in its 
selection of schools and students within those schools. As a result, each student who ends 
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up in the TIMSS sample is assigned a sampling weight'* that must be incorporated into 
statistical analysis for proper inference. 


Both of these features of TIMSS Advanced, plausible values and sampling weights, must be 
used appropriately to obtain proper estimates of achievement on TIMSS Advanced for 
subgroups, and corresponding standard errors. For instance, to estimate the mean TIMSS 
Advanced Math score for AP Calculus BC students, the plausible values and sampling 
weights must be incorporated. 


While there are various statistical methods that can incorporate the plausible values and 
sampling weights, in our analyses, we used the same methods that were used operationally 
in TIMSS Advanced (LaRoche and Foy 2016). The accuracy of our implementation, coded 
using R software, was verified by reproducing various achievement tables published by the 
TIMSS & PIRLS International Study Center. 


Estimating Correlations Among TIMSS and AP Exams 


Given the similarity in content between AP and TIMSS Advanced exams that cover the 
same subject (e.g., TIMSS Advanced Math and AP Calculus BC), it stands to reason that 
student achievement on the two exams would be correlated. Because the TIMSS exams are 
scored using plausible values, calculating the correlations among the exams isn’t 
straightforward. However, “linking” the exams provides a way to quantitatively study this 
relationship. To accomplish this linking, we used a version of the calibrated projection linking 
method proposed by Thissen et al. (2011). 


Unlike TIMSS Advanced, AP does not use Item Response Theory (IRT) for test scoring. 
Consequently, it was necessary to perform IRT item calibrations for each AP subject test. 
These calibrations were performed using the entire population of AP Exam takers, for each 
subject test. For example, for AP Calculus AB, over 250,000 students were used in the 
calibration. 


We conducted the “calibrated projection linking” using a two-dimensional confirmatory item 
response theory (IRT) model. For the present linking, the two dimensions correspond to 
achievement on the AP Exam (8p), and achievement on TIMSS Advanced (81a). The model 
allows a student to receive a distinct score estimate for each dimension, while 
simultaneously estimating the correlation between the two dimensions. This correlation 
estimate helps to quantify the similarity between the AP and TIMSS Advanced Exams. 


Characteristics of the Students in the Study 


This chapter describes the characteristics of the AP—TIMSS Advanced study samples with 
respect to student demographics and family background. The information summarized in 


13. As with the questionnaire analysis, “house weights” were used. 
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this section is drawn from TIMSS Advanced 2015 Student Questionnaire, Advanced 
Mathematics, and TIMSS Advanced 2015 Student Questionnaire, Physics. Descriptions and 
comparisons are first given for the calculus students, followed by descriptions and 
comparisons for the physics students. 


Student Demographics 


As shown in Table 4, females and males were almost equally represented in the AP 
Calculus AB and the non-AP Calculus groups. For AP Calculus AB, 49.2% of the group was 
female and 50.8% was male. The AP Calculus BC group had a distribution favoring males, 
with 43.6% female and 56.4% male. The distribution for the non-AP group was the opposite 
of the AP Calculus AB group, with 50.8% of the group female and 49.2% of the group male. 


The proportion of Hispanic or Latino students in the AP Calculus AB group was 20.9%, more 
than double the proportion of Hispanic or Latino students in the AP Calculus BC group, 
which was 8.1%. In the non-AP group, 12.6% of the students reporting that they were 
Hispanic or Latino. Students were asked about race separately from ethnicity: Which of the 
following best describes you? For the AP Calculus AB group, 75.4% of the responded 
White, 7.0% Black or African American, 13.8% Asian, 1.8% American Indian or Alaskan 
Native, and 1.0% Native Hawaiian or Pacific Islander. These proportions were very similar to 
the racial distribution of the non-AP Calculus group. Compared to the other groups, the AP 
Calculus BC group had a smaller percentage of students reporting their race as 
Black/African American (2.9%) and a larger percentage reporting their race as Asian 
(28.4%). 


Table 4: Demographic Characteristics of Calculus Groups (Percentage) 
| 


Characteristic AP Calculus AB AP Calculus BC Non-AP Calculus 
Females 49.2 43.6 50.8 
Males 50.8 56.4 49.2 
Hispanic/Latino 20.9 8.1 12.6 
White 75.4 74.4 74.7 
Black/African American 7.0 2.9 6.3 
Asian 13.8 28.4 15.8 
American Indian 1.8 4.6 1.7 
Pacific Islander 1.0 0.5 1.6 


Note: Percentages in racial categories may not sum to 100 as students were allowed to check more than one race. 


20 Oo CollegeBoard 


Table 5 shows similar demographic information for the Physics groups. For AP Physics 1, 
45.5% of the group was female and 54.5% was male. The AP Physics C:M group had a 
distribution that favored males, with 24.1% female and 75.9% male. The distribution for the 
AP Physics C:EM group also favored males, but to a lesser extent. The non-AP Physics 
group also favored males, with 40.4% female and 59.6% male. 


The proportion of Hispanic or Latino students in the AP Physics 1 group was 23.6%, 
approximately twice the proportion of the other physics groups. Both the AP Physics 1 and 
the non-AP Physics groups had a larger Black/African American population. For all of the 
physics groups, the percentage of students who self-reported their race as White was 
between 64.8% and 74.2%. 


Table 5: Demographic Characteristics of Physics Groups (Percentage) 
| 


Characteristic AP Physics 1 AP PhysicsC:M AP PhysicsC:EM Non-AP Physics 
Females 45.5 24.1 30.2 40.4 
Males 54.5 75.9 69.8 59.6 
Hispanic/Latino 23.6 11.4 12.7 12.0 
White 69.2 72.1 74.2 64.8 
Black/African American 12.1 4.8 1.3 10.4 
Asian 18.2 23.2 22.3 23.3 
American Indian 1.7 0.7 0.4 2.0 
Pacific Islander 1.0 1.4 0.9 1.5 


Note: Percentages in racial categories may not sum to 100 as students were allowed to check more than one race. 
Primary Language 


Students were asked to respond to two questions related to the primary language spoken at 
home: How often do you speak English at home? What language do you speak at home 
(other than English)? Students responded to the first question on a four-point Likert scale. 
As shown in Table 6, for the calculus groups 86.6% of the AP Calculus AB group indicated 
that English was at least Almost Always spoken at home. This combined percentage was 
not statistically significantly different from the AP Calculus BC group (90.1%) or the non-AP 
Calculus group (92.4%). 
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Table 6: Calculus: English Spoken at Home (Percentage) 
SSS SSS SSS SSSSSSSSSee—————————————E——————————— ea] 


AP Calculus AB AP Calculus BC Non-AP Calculus 
Always 70.3 73.7 77.3 
Almost Always 16.3 16.4 15.1 
Sometimes 10.0 8.1 5.4 
Never 3.4 Ls 2.3 


Students were asked about a second primary language (“always’” English students are 
recorded as “not applicable” in Table 7). Over 30% of the AP Calculus AB group indicated 
that they spoke Spanish or some other language at home. Similarly, 25.6% of the AP 
Calculus BC group and 23.6% of the non-AP Calculus group spoke Spanish or another 
language at home, although the percentage of Spanish speakers was less in AP Calculus 
BC than in AP Calculus AB. 


Table 7: Calculus: Spanish or Other Language Spoken at Home (Percentage) 
E__—— | 


AP Calculus AB AP Calculus BC Non-AP Calculus 
Not applicable 69.2 74.4 76.4 
Spanish 15.4 4.3 8.3 
Other 15.4 21.3 15.3 


The physics groups had results that were very similar to calculus for their primary language 
spoken at home. As shown in Table 8, 84.6% of the AP Physics 1 group indicated that 
English was at least Almost Always spoken at home. This combined percentage was higher 
for the AP Physics C:M group (86.6%) and the non-AP Calculus group (88.9%), and slightly 
lower for the AP Physics C:EM group (83.7%). 


Table 8: Physics: English Spoken at Home (Percentage) 

a es) 
How often do you 
speak English at 


home? AP Physics 1 AP PhysicsC:M APPhysicsC:EM Non-AP Physics 
Always 65.6 72.9 70.3 68.6 
Almost Always 19.0 13.7 13.4 20.3 
Sometimes 12.3 8.9 9.9 8.7 
Never 3.1 4.6 5.8 2.4 


When asked about a second primary language (see Table 9), 34.6% of the AP Physics 1 
group responded that they spoke Spanish (18.5%) or some other language at home 
(16.1%). For the other physics groups, between 25% and 30% of the students indicated that 
they spoke Spanish or another language at home. 
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Table 9: Physics: Spanish or Other Language Spoken at Home (Percentage) 

_———— ee ae eS 
What language do 
you speak at home 
(other than English)? AP Physics 1 AP PhysicsC:M APPhysicsC:EM  Non-AP Physics 


Not applicable 65.5 73.6 70.8 69.2 
Spanish 18.5 7.9 6.6 7.3 
Other 16.1 18.6 22.5 23.1 


It is important to note here that this represents a student population committed to completing 
the most advanced mathematics and science coursework offered in U.S. high schools. In 
general, more than one-fourth of these advanced students are in a home that at least 
sometimes speaks a language other than English. The percentage of students who speak 
Spanish at home is also noteworthy, as it represents 15.4% of the AP Calculus AB group 
and 18.5% of the AP Physics 1 group. 


Socioeconomic Status 


In international assessments, the socioeconomic status (SES) of students is often measured 
as a combination of parent characteristics and possessions in the home.'* In the TIMSS 
Advanced survey, educational attainment of parents was demarcated along the lines of high 
school, postsecondary, and post-baccalaureate education. Students were asked to consider 
both parents on a scale ranging from completing less than high school education to earning 
a graduate degree. 


Students in the most advanced AP classes hail from highly educated families. For the 
calculus groups, parents of the AP Calculus BC had the highest educational attainment with 
34.4% of students’ mothers and 39.2% of fathers earning a master’s or doctoral degree. The 
corresponding figure for the U.S. population as a whole over the age of 25 is 15.0%.'° The 
AP Calculus AB group’s parents had the lowest educational attainment, with 33.7% of 
students’ mothers and 34.2% of students’ fathers completing a high school education or 
less.16 


14. Jan-Eric Gustafsson, Kajsa Yang Hansen, and Monica Rosen, “Effects of Home Background on Student Achievement in 
Reading, Mathematics, and Science at the Fourth Grade,” Chapter 4 in TIMSS and PIRLS 2011 Relationships Report, pp. 
181-287. 


15. “Educational Attainment in the United States: 2014”. U.S. Census Bureau. 
16. Response options were slightly different between the SPSS data file and the actual questionnaire (e.g., Asssociate’s 


degree was “short-cycle tertiary”). Questionnaire wording included stepmother or female legal guardian and stepfather or 
male legal guardian. 
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Table 10: Calculus: Parents’ Education (Percentage) 


What is the highest AP Calculus AB AP Calculus BC Non-AP Calculus 
level of education 

completed by your...? Mother Father Mother Father Mother Father 
No high school diploma 10.7 11.6 5.1 6.4 5.6 6.1 
High school graduate 23.0 22.6 15.8 15.7 16.7 20.3 
Associate degree 15.1 9.2 12.4 6.5 15.4 9.5 
Bachelor’s degree 29.7 28.9 31.3 28.2 34.2 29.8 
Graduate Degree 

(Master, Doctorate, or 18.9 22.6 34.4 39.2 24.4 27.7 
Professional) 

| don't know 2.5 5.1 1.1 3.9 3.7 6.5 


For the physics groups, the AP Physics C:EM had the highest educational attainment with 
41.6% of students’ mothers and 52.7% of fathers earning a master’s or doctoral degree. The 
AP Physics 1 group had students with parents who had the lowest educational attainment, 
with 32.1% of students’ mothers and 33.7% of students’ fathers completing a high school 
education or less. Given that students in AP Calculus AB and AP Physics 1 courses 
complete coursework that could earn them college credit, a large proportion of students in 
these courses may exceed the educational attainment of their parents. 


Table 11: Physics: Parents’ Education (Percentage) 

—————SSS SS a 
Whatis the highest = ap physics 1 AP Physics C:M AP PhysicsC:EM Non-AP Physics 
level of education 
completed by 


your...? Mother Father Mother Father Mother Father Mother Father 
No High School 12.7 10.3 5.0 3.9 4.5 2.6 8.4 7.6 
Diploma 

High school graduate 19.3 23.4 15.5 10.5 8.6 11.2 20.1 20.7 
Associate degree 10.5 8.9 8.0 74 red 1.9 11.5 7.7 
Bachelor’s degree 33.7 27.9 40.9 35.7 33.2 26.2 31.7 31.0 


Graduate Degree 
(Master, Doctorate, or 21.3 25.5 28.5 39.7 41.6 52.7 25.3 28.1 
Professional) 


| don't know 2.4 4.0 2.0 2.7 4.4 5.4 3.1 4.9 
For the two AP Calculus groups and the three AP Physics groups, between 45% and 65% of 
the students report having enough books in their home to fill at least two bookcases. The 


non-AP Calculus and non-AP Physics groups have responses in this same range of 
percentages. 
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Table 12: Calculus: Books in Home (Percentage) 
a ee 
About how many books are there 


in your home? AP Calculus AB APCalculus BC  Non-AP Calculus 
None or very few (0-10 books) 7.2 4.7 5.4 
Enough to fill one shelf (11—25 books) 15.2 8.2 13.5 
Enough to fill one bookcase (26-100 31.4 29.2 26.5 
books) 

Enough to fill two bookcases (101-— 20.1 24.4 24.6 

200 books) 

Enough to fill three or more 26.0 33.5 29.9 


bookcases (more than 200) 


Table 13: Physics: Books in Home (Percentage) 
a a a aa ee | 


About how many books are AP Physics AP Physics Non-AP 
there in your home? AP Physics 1 C:M C:EM Physics 
None or very few (0-10 books) 8.2 3.4 1.9 6.2 
Enough to fill one shelf (11-25 12.8 8.6 6.6 13.0 
books) 

Enough to fill one bookcase 29.7 24.9 24.3 30.1 
(26-100 books) 

Enough to fill two bookcases 24.0 27.1 27.3 23.0 
(101-200 books) 

Enough to fill three or more 25.3 35.9 39.8 27.7 


bookcases (more than 200) 


Access to digital devices in the home is a potential indicator of several student 
characteristics. Due to the expense of purchasing and maintaining digital devices it is 
partially related to SES. In addition, however, access to various technology platforms may 
represent student familiarity with the internet and access to resources that could support 
increased educational attainment. Responding to a series of general questions (data not 
shown), over three quarters of students in all calculus and physics groups indicated that 
they have at least seven or more digital information devices in their homes. 


Of course, digital devices that are shared by an entire family may limit access. Two items 
that examined this issue further asked students whether they have their own digital devices 
along with other possessions. Among all calculus and physics groups, over 90% of students 
had their own smartphone, over 80% their own computer, and approximately 50% their own 
car. Given that AP Calculus requires the use of a graphing calculator on some sections of 
the exam, it could be a concern that up to 15% of AP Calculus students do not own a 
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graphing calculator. Interestingly, only 5% of AP Physics C students do not own a calculator; 
however, for the AP Physics 1 and non-AP groups, almost 20% of students donot own a 
graphing calculator. 


Table 14: Calculus: Digital Devices and Other Personal Possessions (Percentage) 
SSS SS SS 


Do you have any of these things? AP Calculus AB APCalculus BC  Non-AP Calculus 


Your own computer 81.4 85.0 80.1 
Your own tablet 38.7 38.0 39.0 
Your own smartphone 93.7 91.2 92.4 
Your own graphing calculator 84.9 87.6 85.5 
“oe (e.g., PlayStation, 80.6 774 766 
Study desk/table for your use 83.5 89.2 85.4 
Your own room 88.1 91.0 91.2 
Your own car 53.7 50.3 56.2 


Table 15: Physics: Digital Devices and Other Personal Possessions (Percentage) 
————————————————S a a a as) 


Do you have any of these AP Physics AP Physics Non-AP 
things? AP Physics 1 C:M C:EM Physics 
Your own computer 86.1 85.8 87.1 82.7 
Your own tablet 33.9 33.7 39.0 37.7 
Your own smartphone 93.8 91.6 95.0 93.8 
Your own graphing calculator 81.4 94.6 95.2 80.2 
aes i oe 74.4 84.5 79.1 78.0 
Study desk/table for your use 83.6 86.8 89.1 84.7 
Your own room 85.1 91.1 90.9 88.6 
Your own car 46.4 51.6 51.8 48.3 


Students also responded to survey items on weekly time commitments—specifically, 
students reported the time they spent inside and outside of school studying math or science, 
if they had a job during the school year and, if they did, the weekly time devoted to that job. 


In mathematics classes, both AP Calculus groups indicated that they spent, on average, 
close to five hours per week in class; the non-AP Calculus group spent about four hours per 
week in class. The difference between the non-AP group and the two AP groups is about 56 
minutes, longer than a single class period in many high schools. The weekly time spent on 
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mathematics outside of class ranged from 3 hours, 15 minutes for AP Calculus BC to 2 
hours, 43 minutes for AP Calculus AB, to 1 hour, 32 minutes for the non-AP Calculus group. 
As the combined time figures show, non-AP students spend dramatically less time each 
week, at least two hours, engaged with mathematics than AP students. The difference is 
both statistically and substantively significant. Over a 36-week course, the shortfall would 
accumulate to at least 72 hours. 


Table 16: Calculus: Time Spent Per Week on Mathematics, Mean (Standard Error) 
in Hours: Minutes 
sSSSSSS——SSS——SSSSSSSSSSSSSSSSSSSSSS552555SSSS55255SSS585253£QQqSSSSSSSSSSzqQqSSSSsSSSSsSscz 


Weekly Time Spent AP Calculus AB APCalculus BC  Non-AP Calculus 
4:55 4:55 : 
In Mathematics Class oe 
(0:08) (0:10) (0:09) 
2:43 3:15 1:32 
On Mathematics Outside of Class 
(0:09) (0:16) (0:05) 
7:37 : : 
Combined Time : ae ae 
(0:12) (0:21) (0:12) 


In physics, the three AP Physics groups responded that they spent between 4 hours, 30 
minutes and 4 hours, 47 minutes per week in class. The non-AP Physics group spent 3 
hours, 25 minutes per week in class. The time spent on physics outside of class ranged from 
just over two hours per week for AP Physics C:M and AP Physics C:EM, to 1 hour, 19 
minutes per week for the non-AP Physics group. AP Physics 1 students spent just under 2 
hours per week on physics outside of class. Similar to the calculus groups, the non-AP 
students spent less time engaged with physics either in or outside of class. 


Table 17: Physics: Time Spent Per Week on Physics, Mean (Standard Error) in 
Hours: Minutes 
—SES>E>EL  _SESEEEE>>EE>>>E>E>E>E>E—>———_ SSS Sass] 


AP Physics AP Physics Non-AP 

Weekly Time Spent AP Physics 1 C:M C:EM Physics 
In Physics Class 4:31 4:30 4:47 3:25 
(0:08) (0:12) (0:17) (0:10) 
On Physics Outside of Class 1:53 2:06 2:05 1:19 
(0:07) (0:09) (0:15) (0:08) 
Combined Time 6:25 6:36 6:52 4:45 
(0:13) (0:18) (0:26) (0:13) 


When not attending school or studying school subjects, teenagers may engage in paid 
employment. Around the world, working during the school year is rare for TIMSS Advanced 
students, and it is especially unusual to devote more than 10 hours per week to paid 
employment (see Table 18). The two outliers are Norway and the United States. In Norway, 
18% of advanced math students and 15% of advanced physics students report working 
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more than 10 hours per week. In the U.S., 20% of advanced math students and 21% of 
physics students work more than 10 hours per week. 


Table 18: TIMSS Advanced Students Working More Than 10 Hours per Week at a 
Paid Job During the School Year 
SSS SSS SSS SSS 


Country Mathematics (%) Physics (%) 
France : : 
(0.1) (0.1) 
3 2 
vy (0.4) (0.3) 
3 2 
Lebanon (0.7) 0.4) 
Nomer 18 15 
(1.9) (0.9) 
Portugal : : 
(0.3) (0.3) 
Russian Federation : . 
(0.2) (0.3) 
; 5 4 
Slovenia (0.5) (0.6) 
6 5 
ehecee (0.4) (0.5) 
United States os 7: 
(1.2) (1.6) 
International Avg. i : 
(0.3) (0.2) 


Source: IEA's Trends in International Mathematics and Science Study—TIMSS Advanced 2015 Exhibits M4.2 and P4.2. 


Table 19 displays data on employment for students in the study’s analytical groups. 
Approximately 30%—40% of students in advanced mathematics and physics courses had a 
part time job while they were in school. Among all groups, the non-AP Calculus group 
(40.5%) had the highest proportion of students who were employed and the AP Physics C:M 
group had the lowest rate of employment (29.4%). Of those students who were employed, 
the mean hours per week spent at their paid job ranged from a high of 16 hours, 45 minutes 
for AP Physics 1 to a low of 9 hours, 35 minutes for AP Physics C:EM. 
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Table 19: Time Spent Working at a Paid Job During the School Year 


AP AP AP AP AP 
Calculus Calculus Non-AP' Physics Physics Physics Non-AP 

AB BC Calculus 1 C:M C:EM Physics 
Do Not Work (%) 68.1 70.2 59.5 70.4 70.6 67.6 65.4 

(2.2) (4.4) (3.8) (3.7) (2.9) (4.6) (4.0) 
Work During the 31.9 29.8 40.5 29.6 29.4 32.4 34.6 
School Year (%) (2.2) (4.4) (3.8) (3.7) (2.9) (4.6) (4.0) 
Weekly Time 
Spent by Working 14:42 13:19 14:04 16:45 14:12 6335 14:27 
Students (0:34) (0:56) (1:03) (0:46) — (0:41) (1:07) (1:08) 


(Hours:Minutes) 


Achievement 


This section examines scores on the 2015 TIMSS Advanced tests for mathematics and 
physics. First, to illustrate the comparability of the math and physics skills tested, the 
correlations among the relevant TIMSS and AP tests are presented. These results 
complement the qualitative comparisons from the alignment study cited earlier. Then, to 
gain a global perspective, results for AP Calculus and AP Physics—and their non-AP 
counterparts—are presented, along with the scores of other education systems that 
participated in the 2015 TIMSS Advanced. The analysis then turns to how the study’s 
sample groups performed on the content and cognitive subdomains of TIMSS Advanced. 
The section ends with a look at gender differences on the tests. 


The correlations among the AP and TIMSS score scales for the relevant comparisons are 
reported in Table 20. The sample sizes on which these correlations are based are also 
reported. The largest correlations are for Calculus AB and TIMSS Advanced Math, and for 
AP Physics 1, and TIMSS Advanced Physics. The smallest correlation was between AP 
Calculus AB and TIMSS Advanced Math, but even that correlation was large (r=.77). These 
results indicate strong correspondence in each subject area regarding students’ 
performance across the TIMSS and AP Exams. These results also complement the more in- 
depth previously released alignment study— TIMSS Advanced 2015 and Advanced 
Placement Calculus & Physics—A Framework Analysis (Lazzaro et al. 2016). 
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Table 20: Correlations Among AP and TIMSS Scores 
SSSR SS a Sa ae ee ee ee nr) 


TIMSS Math 
AP Exam n r 
Calculus AB 1,301 93 
Calculus BC 395 77 


TIMSS Physics 


n r 
Physics 1 747 91 
Physics C:M 469 .80 
Physics C:EM 141 .86 


The TIMSS Advanced Mathematics scores for participating education systems and AP 
Calculus groups are shown in Figure 1. The United States performed in the upper half of 
participating countries, scoring almost the same as the Russian Federation and behind 
Lebanon. The highest-performing system was the Russian Federation Intensive Course 
group, which required more than six hours a week of mathematics instruction. The lowest- 
performing countries on the TIMSS Advanced Mathematics were Sweden and Italy. 


The scores for the current study’s calculus groups are depicted by the final three bars on the 
right side of the figure. Of these three groups, the non-AP Calculus group performed at a 
level comparable to Sweden and under the overall performance of the United States. The 
performance of the AP Calculus AB group was comparable to overall performance of the 
United States. The AP Calculus BC group’s performance exceeded all participating 
education systems’s national average scores and that of the other AP Calculus groups. 
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Figure 1: TIMSS Advanced Mathematics Scores by Education System and AP 
Groups 
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The TIMSS Advanced Physics scale scores for participating education systems and AP 
Physics groups are shown in Figure 2. The United States performed in the bottom half of 
participating countries, just ahead of Lebanon. The highest-performing country was Slovenia 
and the lowest-performing countries were Italy and France. The physics scores for the 
sample groups selected for this study are shown on the right side of the figure. Of these four 
groups, the non-AP Physics and AP Physics 1 groups performed at a level comparable to 
Lebanon. The AP Physics C:M group’s performance was comparable to the Russian 
Federation and Norway. The AP Physics C:EM group’s performance exceeded all 
participating education systems and AP Physics groups. 
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Figure 2: TIMSS Advanced Physics Scores by Education System and AP Groups 
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Tables 21 and 22 display change in test scores over time. The U.S. last participated in 
TIMSS Advanced in 1995. The change in test scores from 1995-2015 is presented in the far 
right column of both tables. Also shown is each participating system’s coverage index, 
which, as described in section 2, is a measure of selectivity. As a reminder, the coverage 
index represents the percentage of the relevant age cohort enrolled in advanced math and 
physics courses; the total enrollment in those courses constitutes the TIMSS Advanced 
target population. In the U.S., the age cohort is considered all 18-year-olds as of July 1, 
2015, with the coverage index representing the percentage enrolled in advanced math or 
physics courses in 2015. 


Compared with their results in 1995, TIMSS Advanced scores for all education systems in 
2015 were apparently lower (see Table 21). The U.S. coverage index in advanced 
mathematics increased from 6.4% in 1995 to 11.4% in 2015. The apparent U.S. decline of 
12 scale score points was not statistically significant. Of the three nations with statistically 
significant declines, France and Italy both increased the proportion of students in advanced 
math. Sweden’s 71-point decline was despite its coverage index falling from 16.2% to 
14.1%. 
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Table 21: Change in TIMSS Advanced Mathematics Score and Coverage Index, 


1995-2015 


Country 


France 


Italy 


Lebanon 


Norway 


Russian 
Federation 
6hr+ 


Slovenia 


Sweden 


2015 


2008 


1995 


2015 


2008 


2015 


2008 


2015 


2008 


1995 


2015 


2008 


1995 


2015 


Coverage Index 


21.5% 


19.9% 


24.5% 


19.7% 


14.1% 


3.9% 


5.9% 


10.6% 


10.9% 


1.9% 


1.4% 


2.0% 


34.4% 


40.5% 


75.4% 


14.1% 


Average Scale 
Score 


463 
(3.1) 


569 
(3.9) 


422 
(5.3) 


449 
(7.2) 


483 
(9.8) 


532 
(3.1) 


545 
(2.2) 


459 
(4.6) 


439 
(4.9) 


540 
(7.8) 


561 
(7.0) 


549 
(8.2) 


460 
(3.4) 


457 
(4.3) 


478 
(9.3) 


431 


Scale Score 
Change 1995-2015 


-107* 


-61* 


ral 
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(4.0) 


2008 12.8% 412 
(5.6) 


1995 16.2% 502 
(5.2) 


United 2015 11.4% 485 


States (5.2) ae 


1995 6.4% 497 
(7.4) 


Note: * a statistical difference of p <.05 


Source: Adapted from Exhibit M1.4 in TIMSS Advanced 2015 


As shown in Table 22, four systems experienced statistically significant declines in 
advanced physics scores: France, Norway, Russian Federation, and Sweden. The apparent 
16-point difference between 1995 and 2015 was not statistically significant. The U.S. 
coverage index increased from 2.7% in 1995 to 4.8% in 2015. In both math and science, the 
U.S. managed to raise the proportion of students taking advanced math and science 
courses without suffering statistically significant declines in achievement. 


Table 22: Change in TIMSS Advanced Physics Score and Coverage Index, 1995— 
2015 
et 


Scale Score 
Average Scale Change 1995- 

Country Year Coverage Index Score 2015 

France 2015 21.5% 373 96° 
(4.0) 
1995 19.9% 469 
(5.3) 
Italy 2015 18.2% 374 
(6.9) 
2008 3.8% 422 
(7.4) 
Lebanon 2015 3.9% 410 
(4.5) 
2008 5.9% 444 
(3.0) 

Norway 2015 6.5% 507 74 
(4.6) 
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2008 6.8% 534 
(4.1) 
1995 8.4% 581 
(5.5) 

Russian 2015 4.9% 508 i 
Federation (7.1) -38 
2008 2.6% 521 

(10.1) 
1995 1.5% 546 
(10.1) 
Slovenia 2015 7.6% 531 ; 
(2.5) 
2008 7.5% 535 
(2.2) 
1995 38.6% 532 
(13.5) 
Sweden 2015 14.3% 455 
-123* 
(5.9) 
2008 11.0% 497 
(5.3) 
1995 16.3% 578 
(3.7) 
United 2015 4.8% 437 { 
States (9.7) “16 
1995 2.7% 454 


(8.1) 
Note: * a statistical difference of p <.05 


Source: Adapted from Exhibit P1.4 in TIMSS Advanced 2015 


Percentage of Students Reaching International Benchmark 


To provide a more meaningful way to interpret scale scores on the TIMSS Advanced tests, 
the TIMSS International Study Center identifies Advanced, High, and Intermediate 
benchmarks. The TIMSS Advanced tests, in contrast to other TIMSS tests for fourth grade 
and eighth grade, did not establish a Low benchmark given the level of mathematics and 
science content and relative difficulty of the advanced tests. For more info on the TIMSS 
Advanced international benchmarks, see https://nces.ed.gov/timss/timss1 5technotes. 
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Table 23 summarizes the percentage of students in the AP Calculus groups and the United 
States who reached the international performance benchmarks for TIMSS Advanced 
mathematics. The percentages of students in the U.S. groups who reached the Advanced, 
High and Intermediate benchmarks were higher than the international medians for those 
benchmarks, which reflects the group’s overall higher performance. The percentage of 
students in the AP Calculus AB group who reached all three benchmarks was comparable 
to the percentage for U.S. students as a whole. In sharp contrast, over 20% of students in 
the AP Calculus BC group reached the Advanced benchmark, and more than twice the 
percentage of students reached the High benchmark compared to the U.S. as a whole. In 
general, both AP Calculus groups had percentages of students reaching the Advanced and 
High benchmarks that were well above the international median. That was not true for the 
non-AP Calculus group. 


Table 23: Percentage of Students Reaching International Benchmarks in 

Advanced Mathematics 

—EEEe_—_——EEx_—EEEEEE | 
Percentage of Students Reaching Each International Benchmark 


Group Advanced High Intermediate 
(TIMSS Cut Score) (625) (550) (475) 
United States 7 26 56 
(AP + Non-AP) (1.2) (1.6) (2.5) 
AP Calc AB . a es 
(1.5) (2.7) (3.7) 
AP Calc BC = a . 
(4.3) (4.8) (2.8) 
Non-AP : : ss 
(1.2) (3.3) (6.3) 
International Median 2 14 43 


Table 24 shows the percentage of students who reached the TIMSS Advanced international 
performance benchmarks for physics; the last row includes the international median as a 
reference. The percentages of students in the U.S. group who reached the Advanced and 
High benchmarks were comparable to the international medians for those benchmarks, but 
the percentage reaching the Intermediate threshold (39%) fell short of the international 
median (46%). The AP Physics 1 and non-AP Physics groups had lower percentages of 
students meeting each benchmark than the international medians. The percentage of 
students in the AP Physics C:M group was approximately double the international median 
for the Advanced and High benchmarks. Over one-quarter of the students in the AP Physics 
C:EM group reached the Advanced benchmark (27%), and nearly two-thirds (64%) reached 
the High benchmark. 
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Table 24: Percentage of Students Reaching International Benchmarks in 

Advanced Physics 

SSS SS SSS SSS SS ————— 
Percentage of Students Reaching Each International Benchmark 


Advanced High Intermediate 
Group (625) (550) (475) 
United States 5 18 39 
(AP + Non-AP) (0.9) (2.1) (3.3) 
AP Physics 1 : ad 
(0.7) (1.7) (3.6) 
AP Physics C:M a = - 
(2.5) (5.4) (5.7) 
27 64 87 
AP Physics C:EM 
(8.5) (7.3) (5.0) 
Non-AP 3 12 28 
(1.8) (5.9) (7.7) 
International Median 5 18 46 


Content Domain Scores 


The content domains addressed on the 2015 TIMSS Advanced mathematics test included 
advanced algebra, geometry, and calculus. Table 25 summarizes the overall results for the 
United States and the mean scores for the two AP Calculus groups and the non-AP group. 


The highest advanced mathematics subscore for the United States was in calculus, and the 
lowest scores were in algebra and geometry. This pattern held for all U.S. mathematics 
groups including the non-AP Calculus group, not surprising considering that the U.S. is 
unique in organizing curricula addressing the three TIMSS Advanced content domains into 
separate, yearlong courses. Most of the world interweaves algebra, calculus, and geometry 
within advanced math courses, including the advanced math course taken in 2015. U.S. 
students, on the other hand, may not have had extensive exposure to algebra or geometry 
for one or more years. 


The AP Calculus BC group was the highest-performing mathematics group, with content 
subscores that were also the highest for all U.S. groups. The AP Calculus AB group had the 
next highest set of subscores, which were comparable to the overall United States group. 
The non-AP calculus group was the lowest-performing group on the TIMSS Advanced 
Mathematics, by a large margin. Scoring highest in calculus was not the tendency for other 
countries. Both Russian Federation groups had higher subscores on algebra and geometry 
than their overall scores, but they had lower subscores in calculus. Portugal and Slovenia 
were stronger in algebra and weaker in calculus and geometry. 
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Table 25: Advanced Mathematics Content Domain Scores 


_——_———— a ee ET 
Geometry (30 items) 


Country or 
Group 


Russian 
Federation 
intensive 
courses 


Lebanon 
United States 
AP Calculus 


AB 


AP Calculus 
BC 

Non-AP 
Russian 
Federation 
Portugal 
France 
Slovenia 
Norway 


Sweden 


Italy 


Overall 
Advanced 
Mathematics 
Average 
Scale Score 


540 
(7.8) 


422 
(5.3) 


Algebra (37 items) 


Ave. 
Scale 


Score 


556 
(9.0) 


(4.0) 


446 
(4.1) 
422 
(4.1) 
414 
(5.1) 


Difference 
from 
Overall 
Advanced 
Math 
Score 


16 
(3.9) 


(3.6) 
7, 
(1.7) 
6 
(3.3) 
9 
(4.0) 
8 
(3.7) 
10 
(1.9) 
12 
(1.5) 
7 
(1.8) 
14 
(1.1) 
49 
(1.6) 
9 
(1.2) 
8 
(2.2) 


Calculus (34 items) 


Ave. 
Scale 


Score 


513 
(8.0) 


433 
(5.2) 


Difference 
from 
Overall 
Advanced 
Math 
Score 


ey, 
(2.3) 


12 
(2.8) 
19 
(2.9) 
22 
(4.6) 
27 
(5.2) 
9 
(4.4) 
-26 
(1.2) 
6 
(1.4) 
3 
(1.8) 
ap 
(2.0) 
4 
(1.5) 
7 
(1.5) 
11 
(2.7) 


Ave. 
Scale 


Score 


560 
(8.4) 


413 
(5.7) 


Difference 
from 
Overall 
Advanced 
Math 
Score 


20 
(3.2) 


The content domains addressed on the 2015 TIMSS Advanced physics test included 
mechanics and thermodynamics, electricity and magnetism, and wave phenomena and 
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atomic/nuclear physics. Table 26 summarizes the overall results for the United States and 
the mean scores for each of the AP Physics groups and the associated non-AP group. 


The highest physics subscore for the United States was Mechanics and Thermodynamics. 
The other two content domains subscores were lower than the overall scale score. As noted 
in Table 26, the AP Physics C:EM group was the highest-performing group in the study and 
that group’s content subscores are also the highest for all U.S. groups. The AP Physics C:M, 
AP Physics 1, and the Non-AP Physics groups had successively decreasing scores in all 
content subscores, with the Mechanics and Thermodynamics subscore greater than the 
overall mean and the other two content subscores less than the overall mean for all three 
groups. This pattern in content subscores was particular to the U.S. and did not necessarily 
hold for all education systems. For example, Italy, Norway, and the Russian Federation had 
Electricity and Magnetism subscores that were greater than their respective overall mean 
scores. Lebanon and France had Wave Phenomena and Atomic/Nuclear subscores that 


were greater than their respective overall mean scores, and their other two content 


subscores were less than their respective overall mean scores. 


Table 26: Advanced Physics Content Domain Scores 
SSS 
Wave Phenomena 
and Atomic/Nuclear 
Physics (35 items) 


Country or 
Group 


Slovenia 


Russian 


Federation 


Norway 


Portugal 


Sweden 


United States 


AP Physics 1 


AP Physics 
C:M 


39 


Overall 
Physics 
Average 

Scale Score 


531 
(2.5) 


508 
(7.1) 


507 
(4.6) 


467 
(4.6) 


455 
(5.9) 


437 
(9.7) 


412 
(9.8) 


508 


Mechanics and Electricity and 
Thermodynamics Magnetism (27 
(39 items) items) 
Difference Difference 
Ave. from Ave. from 
Scale Overall Scale Overall 
Score Advanced Score Advanced 
541 10 530 -1 
(2.7) (1.6) (4.3) (4.5) 
514 7 515 8 
(6.7) (1.6) (8.0) (2.8) 
503 -5 514 7 
(4.1) (1.7) (5.5) (3.8) 
489 22 431 -35 
(4.8) (3.2) (5.8) (4.5) 
455 0 455 1 
(6.1) (2.7) (6.0) (2.6) 
462 25 379 -58 
(9.6) (3.4) (12.2) (3.9) 
442 30 347 -66 
(9.8) (4.0) (10.6) (5.3) 
534 27 471 -36 


Ave. 


Scale 
Score 


511 
(4.5) 


490 
(7.5) 


507 
(5.2) 


456 
(6.2) 


451 
(6.3) 


431 
(8.7) 


409 
(8.7) 


490 


Difference 
from 
Overall 
Advanced 


-20 
(3.9) 


-17 
(2.1) 


0 
(2.1) 


44 
(5.2) 
-4 
(2.7) 
eT 
(3.0) 
48 
(4.4) 


=17 
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(12.2) (12.4) (4.7) (16.3) (7.5) (10.5) (5.7) 
AP Physics 572 588 16 553 -18 545 -26 
C:EM (13.2) (11.1) (10.4) (17.0) (11.5) (10.7) (10.1) 
Non-AP 405 427 22 336 -69 406 2 
(23.0) (22.0) (5.4) (31.3) (10.4) (23.5) (6.6) 
Lebanon 410 395 -15 399 Wi 431 20 
(4.5) (4.4) (4.7) (5.2) (5.9) (6.8) (5.7) 
Italy 374 376 2 425 51 329 -45 
(6.9) (6.4) (2.6) (6.6) (3.7) (7.9) (2.3) 
France 373 327 -46 339 -34 418 45 
(4.0) (5.7) (3.7) (4.7) (3.8) (4.5) (2.5) 


Note: AP student categories based on AP exam-taking. The category “Non-AP” students excludes all students who took an AP 
course but did not take the AP exam in 2015. 


Cognitive Domain Scores 


For both subjects, three cognitive domains of knowing, applying, and reasoning were 
articulated in the 2015 TIMSS Advanced framework. Each of these three domains is in turn 
described by a set of specific thinking skills and behaviors for math and physics. The 
framework states that while there is some hierarchy across these three Cognitive Domains, 
from knowing to applying to reasoning, each domain is assessed with items representing a 
full range of difficulty. 


In mathematics, tasks can elicit different types of thinking processes or skills, from the recall 
of mathematical terms and algorithms, to the application of mathematics knowledge in 
different contexts, to reasoning that involves further generalization of mathematical 
relationships. Student performance disaggregated by cognitive domain for the participating 
education systems, and the AP Calculus groups, is summarized in Table 27. 


Table 27: Advanced Mathematics Cognitive Domain Scores 
—SSESSESSSS——E—————EE—————————————————————————————————— SaaS 


Country or Group Knowing Applying Reasoning 
Norway 445 459 469 
(4.1) (5.1) (4.4) 
Slovenia 466 465 442 
(3.5) (4.0) (4.0) 
Russian Federation 478 491 484 
(6.7) (6.1) (5:3) 
Russian Federation Intensive 538 544 541 
Courses (8.8) (8.1) (7.2) 
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Portugal 479 476 488 
(3.0) (2.9) (3.5) 
United States 488 480 484 
(5.7) (StS) (G3) 
AP Calculus AB 491 482 486 
(9.4) (9.0) (8.7) 
AP Calculus BC 567 552 559 
(6.3) (5.5) (6.4) 
Non-AP 426 432 437 
(12.3) (2.72) (11.5) 
Sweden 405 434 447 
(4.7) (3.6) (3.9) 
France 475 449 462 
(2.7) (3.4) (3.1) 
Italy 423 425 411 
(5.5) (5.4) (5.9) 
Lebanon 453 529 527 
(4.5) (3.8) (3.9) 


Education systems have relative strengths and weaknesses. France and Lebanon 
demonstrated relatively higher performance in the knowing domain compared to their 
students’ scores for other cognitive domains. Norway and Sweden had relatively higher 
performance in the reasoning domain. The AP Calculus AB and AP Calculus BC groups had 
somewhat consistent performance across the cognitive domains, which is similar to overall 
performance for the United States. The AP Calculus BC group had the highest mean score 
for knowing. The performance of AP Calculus BC group for applying and reasoning, taking 
into account the standard error for each mean, was similar to that of students in Russian 
Federation Intensive Course. The performance of the non-AP Calculus group was low 
across the cognitive domains, and was comparable to the performance of Italy among 
education systems. 


The data summarized in Table 28 indicate that students in all three of the AP Physics 
courses are preforming highly on reasoning tasks. Indeed, students in Physics C:M and 
Physics C:EM are performing among the highest scoring countries in that domain. College 
Board has dedicated considerable time and effort to ensuring that the AP science 
assessments focus on reasoning. AP science courses ask students to analyze and 
synthesize data, to design investigations and formulate hypotheses, and to engage in 
argumentation and justify conclusions based on scientific evidence. This emphasis is 
evident in the AP science and TIMSS Advanced frameworks. 


i oO CollegeBoard 


It is also worth noting that no significant difference is observed between the non-AP group 
and the AP Physics 1 students. Bearing in mind that this is the first year AP Physics 1 was 
offered, the average scores for each of the cognitive domains is surprisingly similar to the 

non-AP group. This is similar to what was observed in the content subscores between AP 

Physics 1 and non-AP. The overall U.S. cognitive domain averages are again elevated by 
the considerably higher scores of AP Physics C:M and AP Physics C:EM students. 


Table 28: Advanced Physics Cognitive Domain Scores 
a SS a SR ae a | 


Country or Group Knowing Applying Reasoning 
Norway 529 484 519 
(4.2) (5.3) (5.7) 
Slovenia 521 543 514 
(4.2) (3.8) (5.7) 
Russian Federation 517 508 493 
(7.5) (7.6) (6.7) 
Portugal 474 452 481 
(4.7) (5.7) (3.9) 
United States 444 420 455 
(9.8) (10.2) (8.8) 
AP Physics 1 418 392 436 
GZ ai) (8.6) (8.9) 
AP Physics C:M 514 490 519 
(11.7) (13.0) (9.9) 
AP Physics C:EM 575 559 ie 
(12.3) (13.1) (13.7) 
Non-AP 411 386 425 
(23.8) (23.6) (21.4) 
Sweden 452 454 450 
(6.0) (6.4) (6.2) 
France 375 358 397 
(3.9) (5.6) (4.2) 
Italy 367 371 375 
(6.6) (7.3) (7.3) 
Lebanon 378 433 375 
(4.7) (5.4) (6.2) 


42 oO CollegeBoard 


Gender Differences 


Given the historical issues of access and participation of women in mathematics and 
science, an analysis of gender differences in student performance on the TIMSS Advanced 
exams was warranted. Table 29 summarizes the mean performance of females and males 
on the TIMSS Advanced Mathematics assessment for the participating countries and the AP 
Calculus groups. The education systems are sorted by the difference in mean performance, 
from positive differences favoring females to negative differences. With respect to the 
participation of students by gender, Slovenia had a female-to-male participation ratio of 
60:40, Portugal was 51:49, Russian Federation was 50:50, and the United States was 
49:51. Lebanon had the greatest participation ratio favoring males at 36:64. 


In comparing mean performance on the TIMSS Advanced Mathematics exam by gender, 
Italy was the only country that had positive differences in performance favoring females that 
were greater than the standard error. Lebanon and Portugal were the only two countries that 
had differences less than the standard error. All of the other participating countries had 
differences favoring males on the TIMSS Advanced Mathematics exam, with the United 
States having the greatest difference among countries favoring males. The performance of 
the AP Calculus AB group stands out from the other United States groups given the 
difference of only 9 scale score points. AP Calculus BC had the highest scoring group of 
female students; however, this group also had the largest difference favoring males, even 
larger than the overall performance for the United States. The non-AP Calculus group had a 
difference in performance favoring males that was similar to the gender difference for the 
United States. 


Table 29: Gender Differences in TIMSS Advanced Mathematics Scores 
eS P| 


Country Females Males Difference 
Italy 427 419 +8 
(6.1) (6.6) (7.5) 
Lebanon 533 531 +2 
(4.8) (3.9) (6.1) 
Portugal 481 483 -2 
(3.0) (3.1) (3.6) 
Russian Federation 480 489 -9 
(6.0) (6.2) (4.3) 
Norway 453 463 - 10 
(5.1) (5.2) (4.8) 
Sweden 424 436 - 13 
(5.1) (4.6) (5.3) 
Russian Federation Intensive 530 549 - 20 
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Courses (9.0) (7.5) (5.2) 
France 449 475 - 26 
(3.1) (3.4) (2.8) 
Slovenia 449 476 -27 
(3.5) (4.9) (4.7) 
United States 470 500 - 30 
(5.3) (6.4) (5.8) 
AP Calculus AB 482 491 =& 
(7.9) (7.9) (12.1) 
AP Calculus BC 539 55 - 36 
(7.3) (7.3) (9.4) 
Non-AP 425 453 - 28 


(11.1) (15.5) (13.6) 


Table 30 summarizes the mean performance of females and males on the TIMSS Physics 
exam for the participating countries and the AP Physics groups. As with Table 29, the 
education systems are sorted by the difference in mean performance, from positive 
differences favoring females to negative differences. With respect to the participation of 
students by gender, no country had a female-to-male participation ratio favoring females. 
France had a female-to-male participation ratio of 47:53, Russian Federation was 42:59, 
and the United States was 39:61. Portugal had the greatest participation ratio favoring 
males at 25:75. 


In comparing mean performance on the TIMSS Physics exam by gender, Lebanon was the 
only country that had a difference in performance favoring females. All of the other 
participating countries had differences favoring males on the TIMSS Physics exam, with the 
United States having the greatest difference among countries favoring males. The 
performance of the AP Physics C:EM and AP Physics 1 had a difference in mean 
performance favoring males that was similar to the gender difference for the United States. 
The difference in mean performance by gender for the non-AP Physics and AP Physics C:M 
groups was less than that for the United States. In general, for the United States the gender 
differences on the TIMSS Physics exam were greater than that for the TIMSS Advanced 
Mathematics exam, although the differences favoring males for six of the seven U.S. groups 
are large and significant. 


Table 30: Gender Differences in TIMSS Physics Scores 
SSS ee SS ————— ——————— | 


Country Females Males Difference 

Lebanon 417 406 +11 
(5.2) (6.4) (8.2) 

Sweden 448 459 -11 
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(6.1) (6.6) (4.9) 
Portugal 456 470 - 14 
(6.2) (5.1) (6.8) 
Russian Federation 498 514 - 16 
(7.9) (7.3) (5.8) 
Norway 489 515 - 26 
(6.0) (4.8) (5:3) 
Slovenia 510 540 - 29 
(6.5) (3.7) (8.6) 
Italy 356 389 - 32 
(7.3) (8.4) (7.8) 
France 354 390 - 35 
(4.2) (4.6) (3.8) 
United States 409 455 - 46 
(11.9) (9.3) (aa) 
AP Physics 1 390 431 -41 
(10.4) (10.4) (11.6) 
AP Physics C:M 488 514 - 26 
(19.5) (11.3) (16.3) 
AP Physics C:EM 539 585 - 46 
(20.7) (13.0) (18.9) 
Non-AP 386 418 - 32 
(27.0) (21.7) (13.1) 


Linking TIMSS and AP: Mean TIMSS Advanced Score by AP Score Groups 


AP students take an exam at the end of each AP course, with scores awarded on a 1 
(lowest) to 5 (highest) scale. Scores of 3 and above are conventionally considered passing 
scores, and many colleges and universities award college credit to students attaining such 
scores. Is there a relationship between AP Exam scores and TIMSS scores? 


Table 31 displays the mean TIMSS Advanced score for AP students, disaggregated by 
those who scored above and below the “3” threshold.'” The differences in TIMSS means are 
large, as shown in the far right column. Consider that the established standard deviation of 
TIMSS Advanced assessments is 100. Four of the five AP group differences are larger than 
100 points, and the smallest difference—83 points in AP Calculus BC—represents more 
than three-quarters of a standard deviation. Such large differences suggest that a score of 3 


17. We would provide the same mean TIMSS Advanced scores for our non-AP groups, but since they have not taken an AP 
Exam, no scores are available. 
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or higher on AP Calculus and Physics Exams does indeed identify students achieving at a 
higher level than those students scoring below 3.'® 


In 2001, Gonzales and colleagues administered the 1995 TIMSS Advanced assessment to 
students taking AP Calculus and AP Physics courses. The researchers also found that 
students with scores of 3 or above on AP Exams performed at significantly higher levels on 
TIMSS Advanced compared to students scoring below 3 on AP Exams. Differences 
between the current study’s methods, which follow TIMSS Advanced protocols, and those of 
Gonzales et al., make comparing results to the current study inappropriate. Only high school 
seniors participated in the current study; the Gonzalez sample included juniors, 
sophomores, and freshmen enrolled in AP. Students took only a single assessment in the 
current study, even if they were enrolled in both AP Calculus and AP Physics courses. In the 
Gonzales et al. study, students taking both AP Physics and AP Calculus courses were 
administered both TIMSS tests. 


Table 31: Mean TIMSS Advanced Score by AP Score Groups < 3 or 23 
a] 


AP (Seniors Only) 
AP Test AP <3 AP 23 Difference 
Calculus AB 427 547 120 
(9.6) (5.2) 
Calculus BC 494 577 83 
(9.9) (6.8) 
Physics 1 371 500 129 
(11.1) (8.6) 
Physics C:M 428 534 106 
(15.4) (9.3) 
Physics C:EM 501 603 102 
(16.4) (14.3) 


Contexts of Learning 


Student Preparation: Advanced Mathematics 


Most schools prepare students for advanced mathematics through a sequence of course 
offerings. Some high schools offer an integrated option in which algebra, geometry, 
trigonometry, and calculus topics are taught in each course. Although, according to National 
Assessment of Educational Progress (NAEP) 12th-grade data and the survey data of the 


18. One key question is whether AP Exam takers differ who are just above or below the cut point. How different are the TIMSS 
scores of AP students scoring 2 from students scoring 3? That question is outside the scope of the current study but may 
be investigated in follow-up work. 
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current study, enrollment in integrated courses never exceeds 5% in any 9-12 grade.'9 In 
the current study’s school survey, school administrators were asked if schools had a special 
program or track that prepared students for advanced mathematics. Approximately two- 
thirds (67.0%) of U.S. students attended schools that had a program designed to prepare 
students for advanced mathematics. 


When do most students take particular math courses? Consider the results from the latest 
federally funded longitudinal survey, the High School Longitudinal Study of 2009 
(HSLS:09).2° Researchers began collecting data on a nationally representative sample of 
more than 21,000 ninth graders in the fall of the 2009-10 school year. Periodic follow-up 
surveys were conducted, including one after the cohort’s graduation in 2013, and a study of 
the sample’s high school transcripts was completed that same year. As ninth graders, 
approximately 5.0% of the students enrolled in Algebra II, 20.5% in Geometry, and 48.7% in 
Algebra |. Starting with Algebra I, the common sequential approach to school mathematics 
in the United States requires four courses before enrolling in an AP Calculus course. Taking 
Geometry or Algebra II as high school freshmen can be considered the next segment of an 
accelerated pipeline leading to calculus no later than the senior year—and for some 
students, even earlier.*' 


Course Taking: Advanced Math Students 


Table 32 summarizes the TIMSS Advanced students’ reporting of the grades in which they 
completed mathematics courses. The most prominent grade-course combination for AP 
Calculus AB students is shaded. It also happens to match the course taking of non-AP 
Calculus students. Over 80% of AP Calculus AB and non-AP Calculus students completed 
Algebra | in Grade 8 or earlier. Over 70% completed Geometry in Grade 9, Algebra II in 
Grade 10, and Precalculus in Grade 11. 


The AP Calculus BC group stands out as the most accelerated group. More than a third 
(36.1%) completed Geometry in eighth grade or earlier, almost half (45.9%) completed 
Algebra Il in ninth grade, and almost half (45.3%) completed Precalculus as a high school 
sophomore. Over half of the AP Calculus BC group (58.6%) completed a calculus course in 
11th grade, meaning that AP Calculus BC in 12th grade was the second year studying 
calculus for many of these students. 


19. Brown, J., Schiller, K., Roey, S., Perkins, R., Schmidt, W., & Houang, R. (2013). The Nation's Report Card: Algebra | and 
Geometry Curricula--Results from the 2005 High School Transcript Mathematics Curriculum Study (NCES 2013-451). 
Washington, DC: National Center for Education Statistics. 


20. Ingels, S.J., Pratt, D.J., Herget, D.R., Burns, L.J., Dever, J.A., Ottem, R., Rogers, J.E., Jin, Y., and Leinwand, S. (2011). 
High School Longitudinal Study of 2009 (HSLS:09). Base-Year Data File Documentation (NCES 2011-328). U.S. 
Department of Education. Washington, DC: National Center for Education Statistics. Retrieved from 
http://nces.ed.gov/pubsearch. 


21. See Table E1 in U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09), Base-Year. 
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Table 32: Course Taking by Advanced Mathematics Students 


a ee ee ee ee Eh 
Grade in which 


Course completed AP Calculus AB AP Calculus BC Non-AP Calculus 
Algebra | Grade 8 or earlier 83.8 93.2 84.6 
(3.2) (1.6) (2.3) 
Grade 9 14.2 5.6 10.3 
(2.6) (1.6) (2.3) 
Geometry Grade 8 or earlier 7.0 36.1 11.1 
(1.2) (5.4) (2.0) 
Grade 9 ales) 48.2 015 
(3.5) (6.0) (3.7) 
Algebra Il Grade 9 15.5 45.9 17.7 
(2.8) (5.4) (4.4) 
Grade 10 75.2 43.4 74.2 
(3.4) (7.0) (StS) 
Pre- Grade 10 6.8 45.3 10.0 
Calculus (1.6) (5.7) (2.4) 
(also called 
Introductory Grade 11 85.8 40.8 759 
Analysis) (2.5) (6.8) (3.9) 
Calculus Grade 11 5.1 58.6 11.1 
(1.3) (6.1) (2.9) 
Grade 12 94.5 83.5 79.9 
(1.3) (2.3) (2.8) 


Note: Table displays modal grade and adjacent grade of enrollment for each course. Modal grade for AP Calculus AB is 
shaded 


Student Preparation: Advanced Physics 


Student preparation in science varies widely depending on the state in which a student 
resides. High school graduation requirements, which in science are less standardized than 
math, account for much of this variation.22 Some states require three years of science to 
receive a high school diploma, while other states require only two years, and in many cases 
the two or three courses are not the same. For example, to earn a Regents diploma in New 
York State, students are required to take three science courses, one Life Science, one 
Physical Science, and one additional Life or Physical Science. In Illinois, only two science 


22. http:/www.ecs.org/high-school-graduation-requirements/ 


23. https://www.hesc.ny.gov/prepare-for-college/your-high-school-path-to-college/regents-requirements.html 
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courses are required, and the state does not mandate the type of science course required 
(e.g., biology, chemistry, physics, etc.). Illinois only provides a very broad statement 
regarding graduation requirements related to science, saying, “[i]n determining course 
offerings, school districts should provide the type of courses that will best meet the needs of 
their students as they pursue various work and study options after leaving high school.”24 


As a consequence of the variation in state requirements, the sequence of science courses 
that a student may take in high school is driven by a mix of district or school requirements, 
local resources, student schedules, and student interest. In the HSLS:09 sample, 34.0% of 
ninth graders enrolled in biology, 19.5% took a physical science course, and 12.4% enrolled 
in earth science. The curriculum of a 9th-grade physics or physical science course can be 
vastly different than a similar course offered at 11th grade, making it difficult to compare how 
each of these courses prepared the student for advanced-level science. Notably, only 1.7% 
of HSLS:09 students enrolled in chemistry as ninth graders, and more than one out of six 
(17.9%) took no science course at all in their freshmen year.*° 


In the current study, as was the case with advanced math, about two-thirds (69.3%) of 
administrators responding to the school survey indicated that their school had a special 
program or track to prepare students for advanced physics. Non-AP Physics students were 
more likely to attend such schools than their peers in AP groups. More than 8 out of 10 non- 
AP students (83.0%) attended a school with a special program or track, compared to 64.6% 
of students in AP Physics 1, 56.1% in AP Physics C:M, and 50.9% in AP Physics C:EM. 


Table 33 indicates that most students in the current study take their first year of biology in 
either 9th or 10th grade. This is especially true for AP Physics C:M, (76.5%) and AP Physics 
C:EM (74.2%) students, suggesting that the more accelerated AP Physics students take a 
traditional biology, chemistry, physics course sequence. AP Physics C:M and AP Physics 
C:EM students are far more likely to take two years of physics, with 79.6% of AP Physics 
C:M students and 89.7% of AP Physics C:EM students taking second year physics in the 
senior year. Those figures compare to only 36.2% of non-AP Physics students and 46.7% of 
AP Physics 1 students. 


Table 33: Course Taking by Advanced Physics Students 
ESS ee ee | 


Grade in which AP Physics AP Physics Non-AP 

Course Completed AP Physics 1 C:M C:EM Physics 
Biology (first Grade 9 BSS) 76.5 74.2 48.1 
year) (6.0) (6.1) (9.0) (9.7) 
Grade 10 32.0 14.7 19.2 27.5 


24. https://www.isbe.net/Documents/grad_require.pdf 


25. Table E11 in U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09), Base-Year. 
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(5.2) (4.5) (8.8) (8.9) 

Chemistry Grade 10 59.0 779 74.0 60.9 

(first year) (5.2) (4.9) (8.3) (9.3) 

Grade 11 31.0 13.3 12.3 20.9 

(5.1) (4.4) (6.8) (6.4) 

Physics Grade 11 36.0 63.6 82.3 35.6 

(first year) (5.6) (6.8) (5.2) (9.6) 

Grade 12 49.4 Is} 4.3 34.9 

(6.1) (5.8) (2.0) (7.7) 

Physics Never 46.9 15.0 3.0 48.9 

(second (5.5) (5.7) (1.9) (8.7) 
year) 

Grade 12 46.7 79.6 89.7 36.2 

(5.1) (5.9) (3.4) (9.2) 


Note: Exhibit displays modal grade and adjacent grade of enrollment for each course. The exception is second year physics, in 
which “never” was the next likely response, after “grade 12,” to taking a second year of physics. No row is shaded for second 
year physics because the “never” and “grade 12” percentages for the AP Physics 1 group does not reach statistical 
significance (p<0.05). Modal grade for AP Physics 1 is shaded. 


AP student categories based on AP Exam taking. The category “non-AP” students excludes 
all students who took an AP course but did not take the AP Exam in 2015. 


AP Calculus and AP Physics students differ as to the broader array of science courses 
available to the AP Physics student. Table 34 shows other AP science courses that students 
have either taken or were taking when they participated in TIMSS Advanced. AP Physics 
C:M and AP Physics C:EM students were far more likely to take another AP science course. 
Most notable is that 46.8% of AP Physics C:M and 45.6% of AP Physics C:EM students 
reported either that they had taken or were currently enrolled in an AP Chemistry course. 


Table 34: AP Science Courses, Taken or Currently Taking 


AP Physics AP Physics Non-AP 
AP Course AP Physics 1 C:M C:EM Physics 
Advanced Placement (AP) 19.2 25.0 29.1 15.7 
Biology (3.1) (3.8) (4.9) (4.2) 
Advanced Placement (AP) 10.0 8.1 6.5 8.7 
Environmental Science (2.3) (1.6) (2.7) (2.6) 
Advanced Placement (AP) 28.6 46.8 45.6 17.2 
Chemistry (3.5) (4.3) (8.7) (3.7) 
Advanced Placement (AP) 64.5 97.9 100.0 0.0 
Physics (2.9) (1.0) (0.0) (0.0) 
Advanced Placement (AP) 8.7 21.4 24.5 4.9 
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Computer Science A or AB (1.6) (3.7) (5.4) (1.5) 


Technology 


This section investigates the role technology plays in advanced mathematics and physics 
courses. Survey data from school administrators, teachers, and students address the 
potential impact of limited technological resources on instruction, the availability of support 
for using technology, how students use technology in calculus and physics classes, and the 
use of the internet in studying calculus. 


Advanced Mathematics 


Historically, mathematics as a school subject has had an interesting relationship with 
technology. Even though computer programming emerged out of mathematics departments 
as an applied field and other mathematics topics such as cryptography and numerical 
analysis use computers extensively, the use of technology in mathematics instruction has 
been limited to the use of scientific or graphing calculators. Over the past two decades, 
advanced mathematics courses in the United States have increasingly integrated the use of 
graphing calculators and many college entrance and national examinations in mathematics 
require the use of graphing calculators. It is within this context that we review survey 
responses to items that reveal the extent to which technology is available to teachers and 
students in mathematics classrooms. 


Table 35: Calculus: School Technological Resources (General) 
LESSEE ———————————————— ay 


Is instruction at your AP Students Non-AP Students 

school affected by a 

shortage or 

inadequacy of the Not at Not at 

following? all Alittle Some _ Alot all A little Some _ A lot 
Technologically 50.7 37.5 8.7 3.1 53.3 22.1 17.3 7.2 
competent staff (4.2) (4.0) (2.0) (1.6) (8.3) (5.2) (6.6) (4.5) 
Audio visual resources 61.4 30.1 7.0 1.5 54.6 32.3 9.5 3.6 


(4.8) (4.3) (1.7) (0.9) (10.9) (10.3) (2.5) (2.9) 
Computer technology 53.0 29.7 14.2 3.1 50.2 27.3 18.4 4.1 
for teaching (5.2) (4.2) (3.2) (1.1) (9.2) (7.5) (5.9) (2.8) 


Note: Responses from school survey. 


School administrators were asked to report whether resource shortages affected instruction 
at their schools. Table 35 displays responses related to technological resources. Over 80% 
of the students in the AP Calculus and 70% of the students in the non-AP Calculus groups 
were enrolled in schools where the administrator indicated “Not at all” or “A little.” Combining 
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the “Some” and “A lot” response categories, however, suggests that there may small but 
noteworthy pockets of schools where shortages have an impact. Problems with acquiring a 
technologically competent staff was viewed by school administrators as affecting instruction 
“Some” or “A lot” for 11.8% of the students in the AP Calculus group and 24.5% of the non- 
AP-Calculus group. Similarly, 17.3% of the AP Calculus students and 22.5% of the non-AP 
group, approximately one-fifth of advanced mathematics students, were enrolled in schools 
in which a shortage or inadequacy of computer technology was viewed as affecting 
instruction “Some” or “A lot.” These results should be taken only as suggestive since these 
items on the school survey did not specifically reference advanced mathematics and physics 
courses when responding to these items. 


In terms of technology used specifically for mathematics instruction (Table 36), 13.8% of 
students in the AP Calculus and 12.7% of students in the non-AP Calculus group attended 
schools affected by a shortage of computer software “Some” or “A lot.” With respect to a 
shortage or inadequacy of calculators, school administrators for 9.8% of students in the AP 
Calculus and 13.6% of students in the non-AP Calculus group indicated instruction was 
affected “Some” or “A lot.” 


Table 36: Calculus: School Technological Resources (Specific to Mathematics) 
ea T| 
Is math instruction at AP Students Non-AP Students 
your school affected 
by a shortage or 


inadequacy of the Not at Not at 

following? all Alittle Some _ Alot all Alittle Some _ A lot 
Computer 48.3 38.0 10.3 3.5 53.5 33.8 11.8 0.9 
Software/Application (5.0) (4.3) (2.3) (1.4) (8.1) (7.4) (4.7) (0.7) 
Calculators 76.0 14.2 5.5 4.3 72.3 14.1 11.9 1.7 


(3.1) (2.8) (2.0) (1.4) (7.1) (5.8) (4.9) (1.1) 
Note: Responses from school survey. 


The teacher surveys also inquired about technology. When asked about the adequacy of 
technological resources, Table 37 shows that 11.1% of the students in AP Calculus groups 
had teachers reporting that a moderate or serious problem existed. A similar proportion of 
students had teachers who saw instructional materials and supplies as a problem (10.1 %). 
Only 4.0% of students in the non-AP group had teachers who felt access to technology was 
a moderate or serious problem, which was far less than the 14.1% of non-AP teachers who 
rated support for using technology as a moderate or serious problem. 
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Table 37: Calculus: Teacher Perceptions of Shortages in Current School 
a a ee ee ee | 


AP Students Non-AP Students 
How severe iseach Nota Minor Moderate Serious Nota Minor Moderate Serious 
problem? problem problem problem problem problem problem problem problem 
Teachers do not 
have adequate 60.6 29.3 7.3 2.8 60.7 26.8 12.3 0.2 
instructional 
mane (5.0) (4.1) (2.0) (1.3) (11.1) (7.9) (7.6) (0.1) 
supplies 
Teachers do not 
have adequate 57.9 30.9 8.0 3.1 57.3 38.7 2.2 1.8 
technological (5.4) (5.4) (2.9) (1.4) 9 (11.2) (11.0) (1.1) (1.2) 
resources 
Teachers do not 
have adequate 50.5 30.5 15.5 3.5 46.8 39.1 12.3 1.8 
support for using (5.5) (4.9) (3.4) (1.4) (11.4) (12.0) (5.8) (1.2) 


technology 


Teachers were asked about their students’ access to computers, tablets, calculators, or 
smart phones. Nearly all (92.6%) students in the AP Calculus group and the non-AP 
Calculus group (98.6%) had teachers who confirmed that students have access to at least 
one of these devices during advanced mathematics lessons. So, in spite of some 
administrators and teachers identifying access to technology as a problem on other survey 
items, student use of such devices is common. 


Table 38 explores how technology is used. The most common use of technology was to 
draw graphs of functions, with 43.5% of students with teachers in the AP Calculus group 
and 41.1% of students with teachers in the non-AP Calculus group using technology in this 
way almost daily. In fact, over 90% of the AP Calculus group and close to 80% of the non- 
AP Calculus group uses technology to graph functions at least once a week. Contrast this 
with students’ experience in the typical undergraduate calculus course in which only one- 
third of college students are allowed to use graphing calculators on course examinations.”6 
As Bressoud (2015) concluded, “Permission to use graphing calculators on exams is one of 
the sharp discontinuities between high school and college calculus” (p. 10). Given the rapid 
comparisons that can be made with functions when using a graphing calculator, it is not 
surprising that this was the most popular use of technology in advanced mathematics high 
school classrooms. Students also use technology to perform numerical integration, with 
38.8% of students in the AP Calculus group and over 25.8% of students in the non-AP 
Calculus group using technology in this way almost daily. In addition to these uses, over 
one-third of students in each group used technology almost daily to manipulate algebraic 
expressions. 


26. See Table on page 9: http://www.maa.org/sites/default/files/pdf/cspcc/InsightsandRecommendations.pdf 
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Table 38: Calculus: Use of Technology During Advanced Mathematics Lessons 
———————————————————————————————————————————————————————————————————————— SS eee 


How often do you AP Students Non-AP Siudents 

have the students do 

the following 

activities on Every or Every or 

computers, tablets, almost Once or Onceor Neveror almost Once or Once or Never or 
calculators, or every twice a twice a almost every twice a twice a almost 
smartphones? day week month never day week month never 
Read the textbook 15.8 17.8 6.4 60.0 18.8 8.4 8.0 64.7 


or course materials 


incdigital format (5.1) (4.3) (2.2) (5.8) (9.0) (4.4) (3.2) (10.8) 
Look up ideas and 6.1 32.5 37.5 23.8 7.5 29.4 19.0 44.2 
information (2.1) (6.0) (3.9) (5.4) (4.8) (10.2) (6.0) (12.0) 
Process and 23.4 32.3 30.7 13.6 25.9 19.9 25.4 28.8 
analyze data (4.7) (4.9) (5.6) (4.1) (9.9) (5.9) (9.6) (11.0) 
Draw graphs of 43.5 50.2 5.4 0.9 414 37.8 17.0 4.0 

functions (5.5) (5.7) (2.0) (0.5) (9.1) (8.3) 7) (3.8) 
Manipulate algebraic 41.5 24.5 193 20.7 33.0 25.9 14.4 26.6 
expressions (5.6) (5.2) (4.9) (5.3) (10.8) (7.5) (8.3) (10.1) 
Conduct modeling 19.7 22.9 39.2 18.2 9.5 15.2 37.7 37.5 
and simulations (3.9) (4.7) (6.4) (4.8) (4.0) (4.8) (10.6) (12.6) 
Perform numerical 38.8 48.8 8.9 3.6 25.8 47.7 1.8 24.7 
integration (4.6) (4.5) (2.6) (1.8) (7.4) (9.0) (1.2) (11.5) 


Students were asked the extent to which they used the internet in their advanced 
mathematics class. Responses to these questions demonstrate the extent to which 
technology permeates students’ interactions with the organization and content of calculus 
courses (Table 39). Approximately three-quarters of the students in the AP Calculus group 
used the internet to find information or tutorials to help solve mathematics problems. The 
proliferation of instructional videos, applets, and discussion boards for mathematics provides 
students an abundance of resources to turn to when they struggle with solving homework 
problems—and many students use these resources. Many internet-based instructional 
resources are accessible on a smartphone. The ease of accessibility underscores that 
students must be informed users of websites, which often vary in quality or accuracy. 


Students also use the internet to access information about the textbook or course 
assignments, communicate with teachers about assignments and course grades, and 
collaborate with other students. Close to half of students in both the AP Calculus and non- 
AP Calculus groups responded that they used the internet to support their course-related 
communication and access to information. 
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Table 39: Calculus: Student Use of Internet for Advanced Mathematics 

Schoolwork 

LSS 
Do you use the Internet to do any of the 
following tasks (including classroom tasks, Non-AP 
homework, and studying outside of class)? AP CalculusAB APCalculusBC Calculus 


Access the textbook or other course materials 48.0 60.0 42.0 

(3.2) (5.4) (3.8) 
Access assignments posted online by my 56.6 55.7 48.3 
teacher (3.7) (6.7) (6.8) 
Collaborate with classmates on mathematics 41.8 A 39.2 
assignments or projects (2.4) (4.3) (3.6) 
Communicate with the teacher 51.0 58.6 52.2 

(2.5) (3.5) (3.3) 
Discuss mathematics topics with other 32.3 33.9 29.2 
students (2.0) (5.5) (2.7) 
Find information, articles, or tutorials to aid in 72.7 79.1 58.2 
understanding mathematics concepts (2.4) (2.3) (3.8) 
Find information, articles, or tutorials to aid in 78.5 80 62.5 
solving mathematics problems (2.5) (3.0) (3.2) 


Technology: Advanced Physics 


Technology and science are interdependent. As outlined in the Science College Board 
Standards for College Success (2009), College Board, “not only values students’ use of 
technology as a tool to practice science, but also believes it is essential for students to begin 
developing an understanding of the complex relationship among science, technology and 
society” (p. 19). This interdependence is critical for the advancement of both science and 
technology, as ongoing development of technology relies on the advancement of science, 
and progress in science depends on ever more advanced technologies to allow science to 
progress. 


In advanced physics classes, most school administrators did not report shortages in 
technology as a significant barrier to providing instruction. As shown in Table 40, 88.9% of 
AP Physics students and 85.0% of non-AP students were enrolled in schools where 
administrators indicated that technologically competent staff was “Not at All’ or only “A Little” 
problem. These same administrators also reported that adequate technology was available 
in their schools and that technology was not an issue that affected instruction. Only 4.5% of 
AP Physics students and 2.5% of non-AP Physics students were enrolled in schools where 
the administrator reported that inadequate computer technology was affecting instruction “A 
Lot.” 
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Table 40: Physics: School Technological Resources (General) 
SSS ———————— ES SS SS SS eee 


Is instruction at your AP Students Non-AP Students 

school affected by a 

shortage or 

inadequacy of the Not at Not at 

following? all Alittle Some _ Alot all Alittle Some _ A lot 

Technologically 56.8 32.1 7.1 4.0 43.9 41.1 12.3 2.7 

competent staff (5.4) (5.2) (2.4) (2.3) (10.2) (9.9) (64) (1.4) 

Audiovisual resources 59.8 31.5 6.7 2.0 39.2 41.6 18.3 0.9 
(7.7) (7.1) (1.9) (1.2) (12.5) (12.8) (9.9) (0.6) 

Computer technology 47.0 30.9 17.6 4.5 32.7 36.7 28.0 2.5 

for teaching (7.7) (7.2) (4.3) (1.9) (12.3) (13.0) (8.4) (1.3) 


Table 41: Physics: School Technological Resources (Specific to Physics) 
a | 
Is physics AP Students Non-AP Students 
instruction at your 
school affected by a 


shortage or 

inadequacy of the Not at Not at 

following? all Alittle Some _ Alot all Alittle Some _ Alot 
Computer 47.8 36.5 11.6 4.0 34.0 47.1 15.2 3.6 
Software/Application (7.8) (7.6) (4.1) (1.9) (12.0) (13.2) (7.2) (2.1) 
Calculators 78.7 14.3 3:2 3.7 67.8 7.1 20.8 4.4 


(4.3) (3.7) (1.8) (1.7) (10.3) = (3.7) (10.7) (2.2) 


Physics teachers were asked similar questions, and their responses paint a similar picture. 
Shortages in technology do not appear to be a serious problem, as 84.3% of AP Physics 
students and 81.1% of non-AP Physics students had teachers who answered “Not at All” or 
“A Little” when asked if shortage or inadequacies in computer software/applications affected 
their physics instruction. It is also interesting to note that only 3.7% of AP Physics students 
and 4.4% of non-AP physics students had teachers that reported a shortage of calculators 
had affected their physics instruction “A Lot.” Advanced physics students are often expected 
to use technology to model, to graph, and to calculate while solving problems, and both 
teachers and administrators report that in all but a few schools this is not an issue. 


There was one notable exception to the overall positive findings. Teachers were asked if 
they have adequate support for using technology: 17.2% of students in AP Physics had 
teachers that reported this was a “Serious Problem.” Interestingly, this only appears to be an 
issue for AP Physics teachers, as only 1.7% of non-AP students had teachers that reported 
support for using technology was a “Serious Problem.” 
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Table 42: Physics: Teacher Perceptions of Shortages in Current School 
SSS SSS SS SS SSS SS = eae 


AP Students Non-AP Students 
How severe iseach Nota Minor | Moderate Serious Nota Minor Moderate Serious 
problem? problem problem problem problem problem problem problem problem 
Teachers do not 
have adequate 43.9 23.9 29.2 3.0 55.2 13.7 24.5 6.6 
instructional 
mane (6.7) (5.1) (4.6) (2.0) (13.2) (4.7) (12.6) (5.6) 
supplies 
Teachers do not 
have adequate 49.8 19.2 16.2 14.9 47.7 36.9 12.9 2.5 
technological (5.7) (4.0) (4.9) (2.4) (12.7) (13.2) (5.9) (1.6) 
resources 
Teachers do not 
have adequate 46.6 25.3 11.0 17.2 54.3 29.2 14.8 1.7 
support for using (6.2) (5.9) (3.1) (3.1) (12.1) (12.6) (7.5 (1.2) 


technology 


Teachers and Teaching 


This section begins with a discussion of STEM teacher shortages. We then summarize the 
experience and qualifications of teachers of advanced mathematics and physics courses. 
The section concludes with an examination of teachers’ confidence in teaching advanced 
mathematics and physics courses and the instructional challenges they may encounter. 


Teacher shortages contribute to the problem of staffing schools with well-qualified teachers. 
Shortages in math and science have been a policy concern for at least the past two 
decades. As reported by Sutcher, Darling-Hammond, and Carver-Thomas (2016), as recent 
as the 2015-16 school year, “42 states and the District of Columbia reported shortages in 
mathematics [teachers]” (p. 5).2” Shortages are not only concentrated in rural schools, but 
also in urban, high-poverty schools (NCES, 2014, 2016) and in some states more than 
others.*® According to the national Schools and Staffing Survey (SASS), shortages of STEM 
teachers peaked in the 1999-2000 school year, affecting about one-third of schools. In 
2011-2012, about 21% of schools reported difficulty filling STEM vacancies.?9 


Not all high schools offer advanced math and science courses. A 2016 analysis by 
Education Week illustrates the regional disparities in physics offerings. About 40% of U.S. 


27. Sutcher, L., Darling-Hammond, L., & Carver-Thomas, D. (2016). A Coming Crisis in Teaching? Teacher Supply, Demand, 
and Shortages in the US. Retrieved from https://learningpolicyinstitute.org/product/solving-teacher-shortage 


28. National Center for Education Statistics, The Condition of Education (Washington, D.C.: U.S. Department of Education, 
2016) https://nces.ed.gov/programs/ coe/pdf/coe_slc.pdf 


29. See Figure 3 in Cowan, Goldhaber, Hayes, and Theobald, “Missing Elements in the Discussion of Teacher Shortages 
(2016, American Institutes for Research). 


57 oO CollegeBoard 


high schools do not offer physics at all. Schools in rural areas are especially unlikely to 
provide physics courses, but a lot of variation exists among predominantly rural states. In 
Alaska and Oklahoma, the percentage soars to 70% of high schools not offering physics, 
but in lowa, New Hampshire, and Maine, only 15% of high schools are without a physics 
course. More than being a uniquely rural problem, the lack of a physics course appears 
correlated with school size. The high schools offering physics enrolled an average of 880 
students; those without physics averaged only 270 students.°° 


Advanced Mathematics 


In the current study’s advanced mathematics sample, survey data are consistent with the 
findings from SASS. When asked how much their school’s capacity to provide instruction 
was affected by a shortage of teachers with subject-area specialization, about two-thirds of 
advanced students were in schools where the school administrator responded not at all. The 
response rates for the other options were: “a little” (24.2%), “some” (7.9%), and “a lot” 
(2.2%), and in advanced physics, “a little” (16.4%), “some” (13.9%), and “a lot” (3.6%). 


Table 43: Calculus: Teachers’ Experience 
are 


Started teaching AP Calculus AB APCalculus BC  Non-AP Calculus 
15 or more years ago 70.9 69.0 81.9 
(5.7) (8.8) (6.0) 
5-14 years ago 19.8 19.6 16.8 
(4.5) (7.0) (5.9) 
Less than 5 years ago 9.3 11.3 1.3 


(4.4) (6.4) (1.1) 


Note: AP student categories based on AP Exam taking. The category “Non-AP” students excludes all students who took an AP 
course but did not take the AP Exam in 2015. 


Source: TIMSS Advanced 2015 teacher survey and College Board 2015 AP datafile. 


Teachers who are assigned to teach advanced mathematics courses, such as calculus, 
typically have greater seniority relative to their colleagues in mathematics departments. 
Table 43 shows when the current study’s teachers started teaching. For the two AP 
Calculus groups, approximately 70% of students had teachers who started teaching before 
2000, 19.8% had teachers who started in 2001-2010, and 9.3% had teachers who started 
teaching after 2011. For the non-AP Calculus group, 81.9% of students had teachers who 
started teaching before 1970-2000, 16.8% had a teacher who started teaching in 2001-— 
2010, and 1.3% had a teacher who had been teaching for less than five years when TIMSS 
Advanced was conducted in 2015. In general, teachers with the most teaching experience, 
over 15 years, taught most advanced mathematics students. 


30. Liana Heitlin, “2 in 5 High Schools Don’t Offer Physics, Analysis Finds,” Education Week, August 23, 2016. 
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Table 44 summarizes the percentage of advanced mathematics students with teachers in 
five different age groups. As suggested in the previous discussion of years of experience, 
the largest age group for the AP Calculus AB group (39.6%) was the category “50 or more 
years.” For the non-AP Calculus group, 50.8% of students had teachers who were between 
40 and 49 years, and 32.1% of students had teachers who were more than 50-years-old. 
Students rarely had an advanced mathematics teachers who was 29 years old or younger. 


Table 44: Calculus: Teachers’ Age 
SSS aaa] 


How old are you? AP Calculus AB AP Calculus BC Non-AP Calculus 
Under 25 years 1.8 3.8 1.1 
(1.3) (3.7) (1.1) 
25-29 years 9.5 6.7 4.9 
(3.8) (4.8) (3.8) 
30-39 years 18.0 21.4 11.2 
(4.4) (7.0) (4.7) 
40-49 years 31.1 36.1 50.8 
(4.5) (7.5) (11.2) 
50 or more years 39.6 32.0 32.1 
(5.9) (10.7) (10.1) 


In addition to an ongoing shortage of mathematics teachers, another concern among school 
administrators is the extent to which teachers are teaching in an area outside of their area of 
specialization. Table 45 shows the percentage of advanced mathematics students with 
teachers who majored in mathematics, physics, or mathematics or science education. For 
the AP Calculus AB group, at least 75% of the students had a teacher with a degree in 
mathematics or mathematics education (teachers who completed a major in mathematics 
and a licensure program in mathematics education may have selected both). For the AP 
Calculus BC group, 83.8% of students had a teacher who majored in mathematics, and for 
the non-AP Calculus group, 85.6% of students had a teacher who majored in mathematics. 
Approximately 10% of the AP Calculus AB and non-AP Calculus students had teachers who 
majored in physics. 


Table 45: Calculus: Teachers’ College Major 


————SSS—_——_S SSS SF 
What was your college major or 


main area(s) of study? AP Calculus AB APCalculus BC  Non-AP Calculus 
Mathematics 77.1 83.8 85.6 

(5.2) (4.8) (8.0) 
Physics 10.1 7.6 12.6 

(2.7) (2.2) (7.5) 
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Education—Mathematics 75.2 75.4 70.8 
(4.3) (6.5) (12.4) 

Education—Physics 1.1 3.5 2.5 
(0.7) (2.0) (2.0) 

Education—Science 0.9 0.6 5.4 
(0.7) (0.7) (5.2) 


Note: Respondents were allowed to select more than one major. 


As shown in Table 46 the highest level of formal education completed by advanced 
mathematics teachers was most often a master’s degree. For students of AP Calculus BC, 
85% had teachers who earned a master’s degree or beyond. In comparison, 70.1% of 
students of non-AP Calculus were taught by a math teacher with a master’s degree and 
69.2% of students in AP Calculus AB had teachers who earned a master’s degree. Given 
that advanced mathematics teachers average over 15 years of teaching experience, one 
might expect that most of these teachers would have pursued a master’s degree during that 
time to contribute to their own professional development. 


Table 46: Calculus: Teachers’ Highest Degree 
SS OO 
Highest level of formal education 


completed AP Calculus AB APCalculus BC  Non-AP Calculus 
Bachelor’s degree 30.8 15.0 29.9 

(6.0) (5.7) (10.5) 
Master’s degree or beyond 69.2 85.0 70.1 

(6.0) (5.7) (10.5) 


Teachers were also asked about their confidence with particular teaching activities (see 
Table 47). Two of the groups expressed the greatest confidence in assessing student 
comprehension of advanced mathematics. More than half (51%) the students in AP 
Calculus AB and 70.9% of students in non-AP Calculus had teachers reporting very high 
confidence in this classroom practice. The AP Calculus BC group had slightly higher 
confidence than the non-AP Calculus or AP Calculus AB groups in providing challenging 
tasks for the highest achieving students’ interest, with 60.4% of students having a teacher 
with such high confidence. The activity in which advanced mathematics teachers said they 
had the least amount of confidence was improving the understanding of struggling students. 
For the AP Calculus AB group, only 30.5% of the students had teachers who indicated very 
high confidence in helping struggling students. In contrast, more than half (55.4%) of the 
non-AP Calculus group had students with teachers who indicated high confidence. 


Recall that teachers of advanced mathematics courses, in general, had extensive teaching 
experience and most had completed a master’s degree. Yet, finding ways to assist 
struggling students still remains a challenge for these teachers, even with students enrolled 
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in the most advanced mathematics courses offered by the school. To some extent, this is a 
challenge that is at the heart of teaching any subject matter as it requires substantial 
knowledge of how students learn mathematics, content knowledge, and understanding of 
the needs of individual students. 


Table 47: Calculus Teachers’ Confidence in Teaching Activities 


(Question: “In teaching advanced mathematics to this class, how would you 
characterize your confidence in doing the following?” Percentage responding 
“very high confidence”) 
SS 


Teaching Activity AP Calculus AB APCalculus BC  Non-AP Calculus 
Providing challenging tasks for the 49.3 60.4 59.2 
highest achieving students’ interest (4.5) (7.9) (9.6) 
Assessing student comprehension of 51.0 53.1 70.9 
advanced mathematics (5.0) (9.5) (7.5) 
Improving the understanding of 30.5 46.6 55.4 
struggling students. (4.2) (9.1) (8.1) 
Developing students’ higher order 45.1 57.8 49.9 
thinking skills (5.3) (8.3) (9.2) 


Teachers and Teaching: Advanced Physics 


As discussed above, there is a well-documented teacher shortage that exists in the United 
States for qualified physics teachers, particularly at the advanced level.*' For the AP Physics 
1 group, 31.3% of students had teachers who started teaching before 2000, 41.9% had 
teachers who started teaching from 2001-2010, and 26.8% had teachers with less than 5 
years’ experience when the 2015 AP and TIMSS Advanced exams were given. Teachers of 
the non-AP Physics group possess even more seniority; about half (49.8%) of the non-AP 
students had teachers who started teaching before 1970-2000, 35.0% had teachers who 
started teaching between 2001 and 2010, and 15.2% had teachers who began after 2011. 
Students in the AP Physics C:M and AP Physics C:EM groups were the least likely to be 
taught by beginning teachers. Only 5.0% of AP Physics C:M students and 5.4% of Physics 
C:EM students had teachers who began teaching after 2011. 


Table 48: Physics: Teachers’ Experience 
——————————————————— a es) 


What year did you start AP Physics AP Physics Non-AP 
teaching? AP Physics 1 C:M C:EM Physics 
Before 1970—2000 31.3 51.0 58.1 49.8 


31. American Association for Employment in Education, Educator Supply and Demand in the United States, AAEE, Columbus, 
OH (2005), available at http:/Awww.aaee.org/pdf/2004fullreportforwebsite.pdf 
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(7.9) (9.0) (11.3) (14.5) 
2001—2010 41.9 44.0 36.5 35.0 
(9.4) (10.0) (12.2) (10.7) 
After 2011 26.8 5.0 5.4 15.2 
(4.1) (3.8) (5.7) (12.5) 


These results are in line with the pattern observed above in math, that it is common to find 
teachers with the most teaching experience, over 15 years, teaching the most advanced 
physics classes. Table 49 reaffirms this finding by reporting the age of teachers of advanced 
physics. The number of teachers who are at least 50 years old serves as an indicator of 
retirements in the near future. Across the four physics groups, 20%—38% of students had 
teachers in that age group. 


Table 49: Physics: Teachers’ Age 
ee = —————————————————— a ee 


AP Physics AP Physics Non-AP 
How old are you? AP Physics 1 C:M C:EM Physics 
Under 25 years 1.9 2.0 0.0 0.2 
(1.4) (2.3) (0.0) (0.2) 
25-29 years 28.7 4.3 5.3 16.2 
(5.0) (3.3) (5.7) (12.5) 
30-39 years 14.5 35.3 25.3 13.3 
(3.9) (8.9) (10.5) (4.7) 
40-49 years 34.7 31.4 33.3 33.1 
(9.2) (9.0) (12.9) (11.0) 
50 or more years 20.1 27.1 36.0 37.3 


(7.1) (7.7) (11.4) (14.8) 


As shown in Table 50, AP students are more likely than non-AP students to be taught by a 
teacher who majored in physics. Almost 6 out of 10 (58.9%) AP Physics 1 students had 
teachers who were physics majors. For AP Physics C:M students, the figure was 53.7% and 
for AP Physics C:EM, it was 57.3%. About 38% of non-AP Physics students had a teacher 
who majored in physics. The overwhelming majority of AP Physics students, 80% or more, 
had teachers who had earned at least a master’s degree (see Table 51). A little more than 
half (56.7%) of the non-AP group was taught by physics teachers with at least a master’s 
degree. 


Table 50: Physics: Teachers’ College Major 
eS SSS SS SSS SS SS SS SS | 


What was your college major AP Physics AP Physics Non-AP 
or main area(s) of study? AP Physics 1 C:M C:EM Physics 
Mathematics 36.0 20.2 27.0 27.7 
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(7.2) (6.8) (8.7) (14.5) 
Physics 58.9 53.7 Si a15 
(9.3) (9.5) (13.8) (12.4) 
Education—Mathematics 24.4 5.1 14.9 8.5 
(4.5) (3.7) (10.1) (6.6) 
Education—Physics 42.3 18.3 25.7 9.2 
(8.4) (6.7) (11.6) (3.4) 
Education—Science 34.9 11.3 13.5 14.0 
(7.4) (5.5) (8.5) (4.5) 


Note: Respondents were allowed to select more than one major. 


Table 51: Physics: Teachers’ Highest Degree 


Highest level of formal AP Physics AP Physics Non-AP 
education completed AP Physics 1 C:M C:EM Physics 
Bachelor’s degree 9.8 18.4 16.2 43.3 
(3.4) (7.0) (4.1) (14.4) 
Master’s degree or beyond 90.2 81.6 83.8 56.7 
(3.4) (7.0) (4.1) (14.4) 


The teachers were asked about their confidence in teaching. Table 52 displays the 
percentage of students with a teacher expressing “very high” confidence, the highest level of 
response, in several instructional activities. Noticeable differences exist between AP Physics 
teachers’ and non-AP Physics teachers’ level of confidence in three key areas. First, only 
18.1% of non-AP students had a teacher with very high confidence in providing challenging 
tasks for high achievers. That compares with 29.5% of AP Physics 1 students, 35.5% of AP 
Physics C:M students, and 45.8% of AP Physics C: EM students. Similarly, non-AP students 
were unlikely (only 8.4%) to have a teacher expressing very high confidence in assessing 
student comprehension of advanced physics. All three groups of AP students had teachers 
with higher rates of confidence; in AP Physics 1 classes, for example, 34.9% of students 
were taught by an advanced physics teacher indicating very high confidence. Finally, only 
18.5% of non-AP students had a teacher expressing the highest level of confidence in 
developing students’ higher order thinking skills compared to at least a third of students in 
the AP groups—and about half (50.1%) of the AP Physics 1 students. 
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Table 52: Physics: Teachers’ Confidence in Teaching Activities 


(Question: “In teaching advanced physics to this class, how would you 

characterize your confidence in doing the following?” Percentage responding 

“very high confidence”) 

SSE 
AP Physics AP Physics Non-AP 


Activity AP Physics 1 C:M C:EM Physics 
Providing challenging tasks for the 29.5 35.5 45.8 18.1 
highest achieving students’ interest (7.8) (8.7) (13.4) (7.3) 
Assessing student comprehension of 34.9 19.2 31.9 8.4 
advanced physics (7.4) (6.5) (12.8) (3.5) 
Improving the understanding of 20.1 19.7 34.7 31.2 
struggling students. (8.5) (7.0) (9.9) (14.5) 
Developing students’ higher order 50.1 33.1 36.1 18.5 
thinking skills (7.9) (8.3) (12.7) (7.6) 
Teaching physics using inquiry 17.8 11.8 17.8 14.4 
methods (5.6) (5.1) (10.2) (6.8) 


The responses to one question were particularly revealing in regard to teachers' more 
limited confidence in teaching physics using inquiry methods. Just 14.4% of students in the 
non-AP physics group and 11.8% to 17.8% of students in the AP Physics groups had 
teachers who described themselves as very highly confident using inquiry methods. Given 
the focus and effort devoted to encouraging inquiry methods in high school science classes, 
some speculation is warranted as to why these percentages are so small. It is possible that 
these teachers are very good at teaching inquiry methods but underestimate their ability to 
teach in this way. It is also possible that the term “inquiry methods” has come to mean a lot 
of different things, and teachers are unclear or unwilling to say that they are highly confident 
to teach physics using inquiry methods. Perhaps, too, history has a say here. As some 
pedagogical reformers have championed inquiry methods for more than a century—and 
have been inevitably disappointed in how little their efforts changed classroom instruction. 


Student Aspirations and Attitudes 


The groups sampled for this study represent students who completed the most advanced 
mathematics and science courses offered in high schools in the United States. Given the 
likelihood that a considerable proportion of these students may pursue STEM intensive 
fields, it is useful to examine their aspirations and attitudes toward mathematics, science, 
and school. 


Advanced Mathematics 


Students who enroll in Advanced Placement courses, and participate in the culminating AP 
Exams, are completing coursework equivalent to introductory college calculus. Students 
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who score a 3 or higher on the AP Calculus Exam, depending on the institution, can earn 
college credit toward a bachelor’s degree. This is one of the primary incentives for students 
to prepare for the end-of-course AP Exam. Given these incentives, and the relatively high 
achievement in mathematics of this group of students, it is expected that most of this group 
would aspire to pursue a college degree. 


Table 53: Calculus: Educational Aspirations 
| 


Highest Expected Degree AP Calculus AB APCalculus BC  Non-AP Calculus 
High School 0.1 0.5 0.2 
(0.1) (0.5) (0.1) 
Associate degree (2-year college 0.2 0.1 0.5 
program) (0.1) (0.1) (0.3) 
Bachelor’s degree (4-year college 25.2 19.8 30.2 
program) (1.6) (3.5) (3.7) 
Master’s degree or professional 48.7 52.5 49.0 
degree (MD, DDS, lawyer, minister) (2.1) (2.8) (3.8) 
Doctorate (Ph.D., or Ed.D.) 25.8 27.0 20.2 
(1.7) (3.5) (3.6) 


As shown in Table 53, more than one-fourth (25.8%) of the students enrolled in AP Calculus 
AB intend to earn a doctorate and 48.7% intend to earn a master’s or professional degree. 
Likewise, for AP Calculus BC, 27.0% plan on earning a doctorate and 52.5% plan to earna 
master’s or professional degree. That is, over three-quarters of high school students 
enrolled in AP Calculus courses not only intend to earn a bachelor’s degree, they also plan 
on going to graduate school. The master’s degree aspirations of the non-AP Calculus 
students are similar to that of the two AP groups (49.0%), but fewer (20.2%) plan on 
doctoral studies. 


Students were also asked, if you plan to continue your education, which area(s) do you 
intend to study? For this prompt, students were instructed to check all areas that were of 
interest. The two most popular areas of study for both AP Calculus groups were Biomedical 
Science and Engineering. For the non-AP Calculus group, the two most popular areas were 
Biomedical Science and Arts and Humanities. Advanced mathematics students were also 
interested in mathematics or statistics as an area of study: 27.1% of AP Calculus AB, 30.8% 
of AP Calculus BC, and 19.8% of non-AP Calculus students expressed interest in this field. 
In general, AP Calculus AB and BC students were more interested in STEM disciplines as 
an area of study than fields such as business, law, or the social sciences. 


Also, worth noting is the limited interest of all advanced mathematics groups in education. 
For the AP Calculus BC group, only 4.1% of the students expressed interest in education as 
a field of study. For the AP Calculus AB group, 6.9% of students were interested in 
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education. For the non-AP Calculus group, 14.2% were interested in education as an area 


of study. 


Table 54: Calculus: Intended Area of Postsecondary Study 
————— aes 


AP Calculus AB APCalculus BC Non-AP Calculus 


Area of Study 


Mathematics or Statistics 


Physics 


Chemistry 


Biological and Biomedical Science 


Engineering and Engineering 


Technologies 


Computer and Information Science 


Education 


Business 


Law 


Social Science 


Arts and Humanities 


Other Science Fields of Study 


Other Non-science Fields of Study 


27.4 
(1.7) 
13:7 
(1.8) 
13.4 
(1.6) 
35.5 
(2.1) 
29.4 
(2.3) 
16.1 
(2.1) 
6.9 
(0.7) 
23.9 
(1.6) 
8.5 
(1.0) 
16.5 
(1.6) 
16.4 
(1.5) 
10.0 
(1.2) 
6.8 
(1.0) 


30.8 
(2.6) 
23.9 
(4.3) 
18.1 
(2.3) 
34.7 
(2.5) 
37.4 
(2.8) 
20.0 
(3.7) 
41 
(1.0) 
23.7 
(4.0) 
7.6 
(1.8) 
17.4 
(3.0) 
19.9 
(3.6) 
10.1 
(2.3) 
10.3 
(2.4) 


19.8 
(2.7) 
13.4 
(2.6) 
13.7 
(3.0) 
30.8 
(2.9) 
17.0 
(2.0) 
13.8 
(1.8) 
14.2 
(1.6) 
26.7 
(3.1) 
7.4 
(2.0) 
23.4 
(3.3) 
26.9 
(3.2) 
10.7 
(1.3) 
7.7 
(1.5) 


Among the lay public in the United States, mentioning mathematics in conversation often 
provokes a negative emotional response, ranging from people who dismiss their own ability 
to engage in mathematical activity to those with an explicit dislike of the subject. Considering 
this context, the attitudes of students enrolled in advanced mathematics courses toward the 
discipline of mathematics is of interest. If any students might view mathematics favorably, it 


should be students pursuing mathematics at the highest level. 
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Table 55 summarizes students’ attitudes toward studying mathematics. Students were 
asked the extent to which they agreed or disagreed with several statements. In general, the 
majority of students in all three groups agreed with the statement, When | do mathematics 
problems, | sometimes get completely absorbed. This prompt reflects both student 
persistence in problem solving and the sense of flow experienced when a challenge 
presented by a task is aligned with students’ mathematical knowledge.** The AP Calculus 
BC group expressed the strongest agreement with this prompt (44.8% selected “Agree a 
lot’). The AP Calculus AB group (34.7%) had the next highest agreement, followed by the 
non-AP Calculus group (27.3%). The group responses to this prompt were similar to 
students’ responses to, mathematics is one of my favorite subjects. All three groups had a 
majority of students who agreed with the statement. Of those who selected “Agree a lot,” the 
AP Calculus BC group had the highest percentage (47.2%), followed by AP Calculus AB 
(41.1%), and then the non-AP Calculus group (26.2%). It is worth noting that, at the group 
level reported here, the psychological aspects of flow (i.e., getting completely absorbed in a 
math problem) seem to be correlated to viewing mathematics as a favorite subject. 


In response to the prompt, / fee! bored when | do my mathematics schoolwork, a plurality of 
students in all three groups agreed a little with the statement. The AP Calculus BC group 
had the largest percentage agreeing a little (46.3%), followed by AP Calculus AB (41.1%), 
and the non-AP Calculus group (39.7%). When combined with students who selected 
“Agree a lot,” the majority of students in all three groups agreed that they feel bored when 
they do their mathematics schoolwork. In contrast to feeling bored, strong majorities of 
students in all three groups disagreed a little or a lot with the prompt, / dread my 
mathematics class. For AP Calculus BC, 80.1% disagreed with this statement (combining a 
little and a lot), followed by AP Calculus AB (71.2%) and non-AP Calculus (58.7%). 


There was fairly strong disagreement across all three groups to the statement, / wish | did 
not have to study mathematics. For AP Calculus BC, 50.2% disagreed a lot; followed by AP 
Calculus AB (46.9%), and non-AP Calculus (34.4%). The statement, /earning advanced 
mathematics does not seem to be a worthwhile exercise produced strong dissent, with 
“Disagree a lot” the modal response of both AP groups, AP Calculus AB (47.8%) and AP 
Calculus BC (57.8%). The modal response for the non-AP group was “Disagree a little” 
(43.6%). 


Table 55: Calculus: Attitudes Toward Mathematics 


aE 
How much do you agree with these 


statements about the mathematics AP Calculus AP Calculus Non-AP 
you are studying? AB BC Calculus 
When | do mathematics problems, | Agree a lot 34.7 44.8 2M xe) 
sometimes get completely absorbed (2.0) (2.8) (2.3) 


32. Csikszentmihalyi, M. (1997). Finding flow: The psychology of engagement with everyday life. Basic Books. 
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| feel bored when | do my 
mathematics schoolwork 


| dread my mathematics class 


Mathematics is one of my favorite 
subjects 


| wish | did not have to study 
mathematics 
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Agree a little 


Disagree a little 


Disagree a lot 


Agree a lot 


Agree a little 


Disagree a little 


Disagree a lot 


Agree a lot 


Agree a little 


Disagree a little 


Disagree a lot 


Agree a lot 


Agree a little 


Disagree a little 


Disagree a lot 


Agree a lot 


Agree a little 


Disagree a little 


Disagree a lot 


45.0 
(1.9) 
16.2 
(1.3) 
41 
(0.7) 
13.8 
(1.3) 
41.1 
(2.0) 
37.2 
(1.8) 
7.8 
(0.9) 
7.5 
(0.7) 
oe 
(2.1) 
36.0 
(1.5) 
35.2 
(2.0) 
41.1 
(2.1) 
29.1 
(1.6) 
17 
(1.9) 
12.2 
(1.3) 
8.4 
ec 
15.4 
(1.4) 
29.3 
(1.7) 
46.9 


41.9 
(2.8) 
10.1 
(2.5) 
3.2 
(1.0) 
11.7 
(2.4) 
46.3 
(3.4) 
29.7 
(4.2) 
12.3 
(3.1) 
2.8 
(0.8) 
eG 
(2.9) 
35.0 
(3.2) 
45.1 
(3.6) 
47.2 
(3.1) 
30.0 
(2.9) 


15.7 
(2.7) 


7.2 
(2.2) 


2.8 
(0.8) 
12.6 
(2.8) 
34.4 
(3.5) 


50.2 


44.5 
(3.6) 
15.4 
(1.8) 
12.7 
(4.0) 
27.6 
(27) 
39.7 
(2.0) 
23.4 
(2.0) 
9.3 
(1.4) 
16.2 
(2.1) 
25.1 
(1.9) 
28.2 
(1.8) 
30.5 
(3.1) 
26.2 
(4.6) 
30.9 
(4.3) 
21.2 
(2.9) 
21.7 
(3.3) 
14.5 
(1.8) 


23.4 
(2.1) 


ZT) 
(2.4) 


34.4 
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(2.2) (2.8) (3.2) 

Learning advanced mathematics Agree a lot 4.4 3.0 8.5 

does not seem to be a worthwhile (0.8) (0.9) (1.6) 
exercise 

Agree a little 11.3 14.0 18.8 

(1.0) (3.4) (3.0) 

Disagree a little 36.5 25.2 43.6 

(1.7) (3.6) (4.8) 

Disagree a lot 47.8 57.8 29.1 

(1.9) (3.0) (2.6) 


In sum, advanced mathematics students expressed positive attitudes toward school 
mathematics and mathematics as a discipline. Most of the students in all three groups feel 
that mathematics is a worthwhile subject and, in fact, indicated that it was one of their 


favorite subjects. 


Table 56 summarizes students’ impressions of the difficulty of the TIMSS Advanced 
Mathematics test compared to other tests they completed in school during the year. Sharp 


differences are evident among the groups. The majority of AP Calculus BC students (54.4%) 
felt that it was easier than other tests they completed. For the AP Calculus AB group, 42.5% 
responded that it was about as hard as other tests. In contrast, students in the non-AP 
Calculus group thought the TIMSS Advanced Mathematics test was harder (38.2%) or much 
harder (33.0%) than other tests. The contrasting responses suggest distinctive differences in 
the norms and expectations for each group in addition to differences in mathematical 
content. 


Table 56: Advanced Math: Student-Reported Difficulty of the TIMSS Advanced 
Test 


—————————E—EEEEEE——————E——————————————————— 
How hard was this test compared 
to most other tests you have taken 


this year in school? AP Calculus AB APCalculus BC  Non-AP Calculus 
Easier than other tests 23.3 54.4 7.3 

(2.7) (3.8) (2.4) 
About as hard as other tests 42.5 32.5 21.4 

(2.2) (2.6) (1.9) 
Harder than other tests 26.0 11.5 38.2 

(1.9) (2.2) (3.2) 
Much harder than other tests 8.2 1.6 33.0 

(1.2) (0.8) (3.2) 
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Advanced Physics 


The same incentives described above for AP Calculus students apply to AP Physics 
students. Chief among them is that an AP Physics student can earn college credit toward a 
bachelor’s degree by scoring a 3 or better on any of the AP Physics assessments. Between 
50%—-55% of them expect to obtain a master’s or professional degree in college. 
Interestingly, there is no discernable difference in the educational aspirations between the 
students that take an AP Physics course and the non-AP Physics students. 


Table 57: Physics: Educational Aspirations 
a a ee ee 


How far in your education do you AP Physics AP Physics Non-AP 
expect to go? AP Physics 1 C:M C:EM Physics 
High School 0.4 0.1 0.0 0.4 
(0.3) (0.1) (0.0) (0.7) 
Associate degree (2-year college 0.1 0.1 0.0 0.1 
program) (0.1) (0.1) (0.0) (1.9) 
Bachelor’s degree (4-year college 22.5 22.4 20.2 22.5 
program) (2.8) (2.9) (3.7) (2.2) 
Master’s degree or professional 52.9 53.0 55.4 52.9 
degree (MD, DDS, lawyer, minister) (2.4) (2.4) (5.9) (2.6) 
Doctorate (Ph.D., or Ed.D.) 24.2 24.5 24.4 24.2 
(1.6) (2.7) (6.6) (3.5) 


AP and non-AP Physics students have different plans when it comes to what these students 
intend to study in college. (Reminder: students were instructed to check all areas that were 
of interest.) AP students were much more likely to consider physics or engineering as a 
course of study than the non-AP Physics students: 41.6% of AP Physics C:EM and 33.0% of 
AP Physics C:M students choose physics compared to only 15.7% of non-AP Physics 
students. For the field of Engineering and Engineering Technologies, 66.0% of AP Physics 
C:EM and 59.9% of AP Physics C:M students considered that as a possible course of study 
compared to 28.1% of non-AP Physics students. The non-AP Physics students were more 
likely to choose Biological and Biomedical Science (33.1%), Social Science (21.8%), and 
Arts and Humanities (20.9%) than their AP counterparts. 


As seen in the advanced mathematics group, there was limited interest by all of the 
advanced physics students for education as a course of study. Only 9.0% of non-AP 
Physics, 6.6% of AP Physics 1, 3.1% of AP Physics C:M, and 2.2% of AP Physics C:EM 
students planned on education as a future course of study. It is true, as noted in the 
mathematics section, that most physics teachers major in mathematics concurrent with (or 
prior to) enrollment in a licensure program; nevertheless, these numbers are very low and 
support the general conclusion that in the U.S. the most advanced science students do not 
plan to teach and are not interested in the formal study of education. 
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Table 58: Physics: Intended Area of Postsecondary Study 
_——EE——————————————————————————————————————— as 


Area of Study 


Mathematics or Statistics 


Physics 


Chemistry 


Biological and Biomedical Science 


Engineering and Engineering 


Technologies 


Computer and Information Science 


Education 


Business 


Law 


Social Science 


Arts and Humanities 


Other Science Fields of Study 


Other Non-science Fields of Study 


Students were also given several statements that elicited their overall attitudes toward 
physics. As one might expect, this group of advanced physics students had generally 


AP Physics 1 
32.8 
(3.2) 
21.0 
(1.5) 
13.9 
(1.7) 
29.0 
(1.8) 
39.9 
(2.3) 
22.4 
(2.9) 
6.6 
(1.9) 
25.0 
(3.8) 
6.6 
(1.0) 
14.2 
(1.7) 
14.1 
(1.4) 
11.2 
(1.4) 


5.2 
(0.8) 


AP Physics 
C:M 
34.6 
(4.7) 
33.0 
(5.2) 
16.6 
(2.6) 
26.5 
(2.8) 
59.9 
(2.8) 
27.2 
(3.5) 
3.1 
(1.2) 
17.3 
(3.1) 
3.6 
(1.1) 
7.1 
(1.5) 
11.2 
(2.1) 
5.8 
(1.4) 
2.0 
(0.9) 


AP Physics 
C:EM 


42.7 
(6.4) 
41.6 
(6.3) 
20.7 
(3.7) 
19.1 
(5.0) 
66.0 
(4.6) 
34.2 
(6.6) 
2.2 
(1.3) 
24.1 
(5.4) 
3.8 
(3.4) 
12.5 
(4.9) 
15.8 
(3.5) 
13.3 
(4.8) 
7.2 
(3.6) 


Non-AP 
Physics 
27.3 
(5.4) 
15.7 
(4.8) 
11.1 
(1.6) 
33.1 
(2.6) 
28.1 
(4.9) 
19.0 
(4.8) 
9.0 
(2.0) 
27.6 
(2.5) 
8.4 
(2.0) 
21.8 
(4.6) 
20.9 
(3.4) 
11.3 
(2.0) 
11.9 
(1.1) 


positive attitudes toward physics, although there are several notable differences between 
AP Physics students and non-AP Physics students (see Table 59). 


Presented with the statement, it is interesting to learn physics laws and principles, 57.0% of 
AP Physics C:EM students agreed a lot while only 23.1% of non-AP students responded 
that positively. AP Physics 1 (36.8%) and AP Physics C:M (41.8%) students also displayed 
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positive feelings toward the statement. Similarly, students were given the statement, / enjoy 
figuring out challenging physics, and 70.6% of AP Physics C:EM, 52.4% of AP Physics C:M, 
36.4% of Physics 1, and 29.6% of non-AP Physics students agreed a lot. It is clear that AP 
Physics C:M and AP Physics C:EM students have the most positive attitudes toward physics 
and express an enjoyment in tackling the challenges of the subject much more so than their 
non-AP counterparts. 


Table 59: Physics: Attitudes Toward Physics 

SSS SS 
How much do you agree 
with these statements 


about the physics you are AP AP Physics AP Physics 

studying? Physics 1 C:M C:EM Non-AP 

It is interesting to learn Agree a lot 36.8 41.8 57.0 23.1 

physics laws and principles (2.4) (5.0) (6.2) (7.1) 

Agree a little 41.1 42.2 35.4 38.8 

(2.8) (3.6) (5.4) (3.3) 

Disagree a little 16.4 ie 6.9 Zou 

(2.0) (2.6) (2.6) (3.6) 

Disagree a lot 7 4.7 0.6 165 

(1.2) (2.3) (0.5) (a4) 

| dread my physics class Agree a lot 10.7 7.4 7A 16.7 

(2.3) (1.8) (2.7) (2.5) 

Agree a little 24.1 20.7 10.0 25.7 

(2.6) (3.4) (2.8) (4.2) 

Disagree a little 37.0 31.3 22.3 30.9 

(2.8) (3.1) (3.2) (2.4) 

Disagree a lot 28.3 40.5 60.6 26.7 

(3.3) (4.3) (5.6) (3.9) 

| enjoy figuring out Agree a lot 36.4 52.4 70.6 29.6 

challenging physics (2.2) (5.0) (5.3) (5.3) 

Agree a little 40.7 oe / 22.1 38.5 

(2.0) (2.1) (4.3) (2.7) 

Disagree a little leo) 11.6 6.3 SHS) 

(1.9) (3.5) (1.9) (3.9) 

Disagree a lot 5.4 oe) 1.1 12.4 

(1.2) (1.3) (0.6) (2.6) 

Physics is one of my favorite Agree a lot 19.1 39.0 54.5 16.3 

subjects (2.3) (5.2) (5.8) (4.2) 
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Agree alittle 27.9 30.1 26.8 21.3 
(3.0) (2.4) (3.9) (2.5) 

Disagree a little 29.7 20.6 12.5 26.5 
(1.8) (2.5) (3.5) (2.8) 

Disagree a lot 23.3 10.3 6.2 35.9 
(3.2) (3.1) (2.2) (4.5) 


The positive feelings of the two advanced AP Physics groups is also evident in students’ 
responses to the prompt, physics is one of my favorite subjects, with 54.5% of AP Physics 
C:EM and 39.0% of AP Physics C:M students agreeing a lot with the statement. The 
divergence of the other two groups becomes clearer in response to this prompt. Only 19.1% 
of AP Physics 1 and 16.3% of non-AP Physics students agreed a lot that physics was a 
favorite subject and, in fact, more than half of AP Physics 1 students (53.0%) and non-AP 
students (52.4%) disagreed a little or a lot. 


Table 60 indicates that some of the group differences in attitudes may be associated with 
how useful students consider their advanced physics courses. AP Physics C:EM students 
were more than twice as likely to agree a lot with the statement, /earning physics will help 
me get ahead in the world, than their non-AP peers. AP Physics C:EM students were also 
almost three times more likely to disagree a lot with the statement, the physics | am studying 
is not useful for my future, than non-AP physics students. Further, 72.7% of AP Physics 
C:EM, 57.3% of Physics C:M, and 45.6% of Physics 1 students, compared to 34.1% of 
non-AP Physics students agreed a lot that doing well in physics will help me get into the 
college or university of my choice. All three groups of AP Physics students were far more 
likely to agree a lot to the prompt: /earning physics will give me more job opportunities. All of 
this reinforces the idea that AP Physics students see the content that they are learning as 
relevant to their overall career goals and future aspirations. 


Table 60: Physics: Valuing the Study of Physics 

a ee ia 
How much do you agree 
with these statements 


about the physics you are AP AP Physics AP Physics 

studying? Physics 1 C:M C:EM Non-AP 

Learning physics will help me Agree a lot 43.4 Biles 64.7 27.6 

get ahead in the world (2.1) (3.9) (5.8) (2.3) 

Agree a little 40.3 37.4 30.6 46.9 

(2.8) (2.5) (6.0) (253) 

Disagree a little few 8.6 4.7 18.1 

(@ie5) (2.1) (1.7) (2.3) 

Disagree a lot Zul Pall 0 7.4 

(0.9) (0.9) (0.0) (Glee) 
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The physics | am studying is Agree a lot 8.7 4.9 2.7 11.6 

not useful for my future (1.7) (1.1) (1.5) (2.2) 
Agree a little 19 15.2 7.8 27 

(1.7) (3.3) (2.5) (2.2) 

Disagree a little 40.1 35.9 29.8 39.1 

(2.1) (2.9) (6.3) (2.5) 

Disagree a lot 32.2 44 59.6 22.3 

(2.7) (3.6) (6.1) (1.9) 

Doing well in physics will help Agree a lot 45.6 Dies ea 34.1 

me get into the college or (2.0) (3.2) (4.6) (2.3) 

university of my choice 

Agree a little Soro 29.2 22.6 41.8 

(253) (2.6) (4.5) (3.7) 

Disagree a little 15.8 10.9 4.0 15.6 

(1.7) (1.6) (1.8) (2.6) 

Disagree a lot By8) 2.6 0.8 S18) 

(1.6) (0.8) (0.6) (2.1) 

Learning physics will give me Agree a lot 42.8 54.0 73.1 24.0 

more job opportunities (3.4) (4.4) (4.7) (3.7) 
Agree a little 36.8 34.4 17.6 44.6 

(3.5) (2.9) (3.9) (3.3) 

Disagree a little 14.8 9.9 7.9 23.7 

(1.5) (2.1) (2.9) (3.2) 

Disagree a lot 5.6 1.7 1.4 hal 


(1.1) (0.8) (0.8) (1.5) 


Implications for Research and Policy 


This study examines student performance on the TIMSS Advanced exams for groups of 
students who were enrolled in Advanced Placement Calculus and Physics courses, and 
other advanced mathematics and science courses. The survey responses from students, 
teachers, and school administrators addressed a number of pertinent factors related to the 
opportunity to learn advanced mathematics and science in the United States, as compared 
to other participating countries. This report also summarized the characteristics, attitudes, 
and aspirations of students who complete Advanced Placement Calculus and Physics, their 
teachers’ background, and teachers’ confidence and challenges in teaching these courses. 


In this section, we elaborate further on the opportunity to learn advanced mathematics and 
science in the United States, and the related patterns in course taking organized by 
secondary schools. Student performance across content domains and dimensions of 


74 oO CollegeBoard 


cognitive demand are discussed with respect to recent U.S. education policy in mathematics 
and science education. We also found students’ attitudes toward these advanced courses, 
and their commitment to coursework, both in and out of class, to be noteworthy. The 
increased role and use of technology in advanced mathematics and science courses are 
unprecedented, especially when compared to the contexts for learning in undergraduate 
mathematics, and we elaborate further on the implications of this use of technology-based 
instructional resources in U.S. high schools. Lastly, the future challenge of sustaining and 
improving student performance in mathematics and science education relies on teachers. 
Given the experience and the age of teachers who teach advanced mathematics and 
science the United States, we discuss the implications for policy in teacher recruitment, 
preparation, and induction. 


Opportunity to Learn 


In the United States, more high school students are taking AP Calculus and AP Physics 
Exams than ever before. In the 2014 AP Report to the Nation over 300,000 students took 
AP Calculus Exams, and over 100,000 students took AP Physics Exams.*° When compared 
with exam participation rates in 2003, this represents an increase in participation in AP 
Calculus of over 75%, and an increase in AP Physics of 85%. In short, student access to 
and completion of advanced coursework in mathematics and science is not only at an all- 
time high in the United States, the increase in student demand for these courses reflects a 
trend that is promising with respect to student preparation for STEM majors and careers. 


In addition, there is clear evidence from the 2014 AP Report to the Nation that low-income 
students’ access to AP courses has increased significantly over the past 10 years. These 
data show that even as AP participation rates have significantly increased, the overall AP 
test scores have remained stable, and in some instances increased. This would suggest that 
increasing student access to advanced level coursework does not require sacrificing the 
rigor of the coursework offered. Further, it is encouraging that the population of students 
participating has become more diverse and the overall AP Exam scores have remained 
stable. Additional evidence of this can be found in the TIMSS trend data, which supports this 
overall conclusion. As stated in section 4, the TIMSS Advanced coverage index for math 
and physics has nearly doubled, from 6.4% to 11.4% for math and from 2.7% to 4.8% for 
physics, from 1995 to 2015. Over this same timeframe there is no statistically significant 
difference in TIMSS Advanced performance results for U.S. students. This is an important 
finding, one that warrants more attention given that the U.S. was the only participating 
country to both increase student access of advanced level coursework without seeing a 
decline in performance results. 


Another way that opportunity to learn may be affected is through content coverage and 
cognitive demand. Not all advanced math and physics classes study the exact same topics 


33. https://secure-media.collegeboard.org/digitalServices/pdf/ap/rtn/1 Oth-annual/1 Oth-annual-ap-report-to-the-nation-single- 
page.pdf 


75 oO CollegeBoard 


or place identical cognitive demands on students. If students have not had an opportunity to 
learn mathematics or science topics, certainly their chance of responding correctly to a 
relevant test item is reduced. Similarly, students who have not been asked to know, to 
apply, or to reason rigorously during classroom instruction cannot be expected to 
demonstrate such skills with sophistication. 


A study of TIMSS 1995 national-level data found that content coverage was significantly 
correlated with the percentage of items answered correctly, explaining between 25% and 
50% of the between-country variance (Angell et al. 2006). In section 4, we reported that 
calculus was the strongest content domain for all of the study’s analytical groups, with 
weaker showings in algebra and geometry. That distribution makes sense considering that 
U.S. students were enrolled in a calculus course when they took TIMSS Advanced, having 
completed courses in algebra and geometry in previous years. Physics students (with 
Physics C:EM students the exception) scored weakest on electricity and magnetism. A 
question demanding further research is whether these relative strengths and weaknesses 
matter for success in college. If America’s top math and science seniors are graduating from 
high school needing review of material taught in earlier grades, and the forgotten material 
may negatively affect success in college, K-12 math and science educators would benefit 
from knowing this. 


Performance of calculus students appeared balanced across TIMSS Advanced cognitive 
domains. In physics, applying was generally the weakest area for U.S. students. For AP 
Physics 1, the performance difference between applying (392) and reasoning (436) was 
statistically significant. Recent changes in the AP Physics curriculum have emphasized 
reasoning, which may partially explain these numbers. Cognitive domains have only recently 
begun to receive attention in international assessments. A question demanding more high 
quality research is whether there is an optimal balance of knowing, applying, and reasoning 
in a yearlong advanced mathematics or science course and whether particular topics benefit 
from stressing one domain over the others. 


Student Attitudes 


As reported in section 5, advanced students in math and science express positive feelings 
toward mathematics and physics. They have high aspirations for earning advanced degrees 
after high school. Many would like to study STEM topics in college and intend to enter 
professional careers that draw on mathematics and science. They also generally express 
enjoyment with studying mathematics and physics; however, responses to one prompt from 
the physics student questionnaire deserve scrutiny. In response to the statement, physics is 
one of my favorite subjects, more than half of non-AP Physics students said they disagree 
either a little (26.5%) or a lot (85.9%). A majority of AP Physics 1 students expressed similar 
sentiments (29.7% disagreed a little and 23.3% disagreed a lot). 


As noted in the introduction, the relationship between student attitudes and learning isn’t 
always clear cut. Myriad factors can influence students’ selection of favorite school subjects, 
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and it may be particularly difficult for some adolescents to develop positive views toward 
challenging coursework. Again, we underscore that the overall picture of student attitudes 
and aspirations is positive. Nevertheless, we wish to convey to teachers and principals of 
advanced students that, despite these students’ high achievement, their contentment with 
math and science classes cannot be taken for granted. 


Time Differences 


In section 4, we found that in comparison to non-AP students AP students devote more time 
to mathematics and science in and out of school. For total time spent each week on math, 
AP Calculus BC students estimated 8 hours, 10 minutes; AP Calculus AB students 
estimated 7 hours, 37 minutes; and non-AP Calculus students estimated 5 hours, 32 
minutes. The shortfall for non-AP students—at least two hours per week compared to 
Calculus AB students and over two and one-half hours compared to Calculus BC—is best 
appreciated when aggregated over a standard school year of 36 weeks. The data suggest 
AP students spend 72—90 more hours engaged with calculus and physics than their non-AP 
counterparts. 


That is a substantial amount of time. The main cause of the discrepancy cannot be 
determined from the data, but student choice in selecting classes (some may have elected 
to take two physics or calculus classes concurrently), along with individual preferences 
about how to spend out of school time, is one possible explanation. 


Students who struggle with calculus or physics should be encouraged to spend more time 
on these subjects. The matter need not be left solely to student discretion. One policy 
intervention that shows promise is known as “double dosing.” Cortes, Goodman, and Nomi 
studied a Chicago program that assigned low-skilled ninth graders to two periods of algebra. 
Employing a regression discontinuity design that allows for causal inferences, the 
researchers found that the program significantly boosted ninth grade test scores and credits 
earned.*4 More importantly, the researchers followed the students through the remainder of 
their high school careers and discovered significant long-term effects, including higher 
graduation and college enrollment rates. Educators of advanced students, especially of 
students for whom advanced courses may be difficult, should investigate double-dosing as a 
scheduling option. 


Technology 


In this study, we found that technology use is somewhat ubiquitous in high school calculus 
and physics courses in the United States. Graphing calculators and Internet browser-based 
software are used by students to graph functions, solve equations, manipulate algebraic 
expressions, and run simulations. In spite of the current use of technology in AP Calculus 


34. Kalena Cortes, Joshua Goodman, and Takako Nomi (2013). “Intensive Math Instruction and Educational Attainment: Long- 
Run Impact of Double-Dose Algebra,” Harvard Kennedy School Faculty Research Working Paper Series, RWP13-09. 
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and AP Physics, teachers have limited access to professional development that supports 
the use of technology in these courses (Lawless and Pellegrino 2007). Most school 
administrators report that AP Calculus teachers have access to and support for the use of 
technology, although access to technology appears to be a moderate or serious problem for 
students of almost one-third of AP Physics teachers in this sample. We would argue that 
teacher and student use of technology in high school calculus and physics is one of the 
most crucial shifts in the use of instructional resources in these courses over the past 20 
years. These findings suggest the need for additional research on how teachers can best 
use technology in instruction in mathematics and science. 


In the United States, when comparing high school calculus to undergraduate calculus 
technology use also represents a major difference in the use of instructional resources. 
Technology is used weekly in high school calculus, but it’s rarely used in postsecondary 
calculus instruction. Even though these differences have been noted by Bressoud (2015) 
and in other studies of postsecondary calculus, there appears to be little movement toward 
greater use of technology in undergraduate mathematics departments, with the exception of 
using online homework tools (e.g., WebAssign) and student use of internet-based 
instructional videos outside of class. These differences in the use of technology in physics 
instruction in high schools and colleges are not as pronounced as they are for calculus. 
Even though education policy does not typically influence undergraduate instruction, a 
research agenda that explicitly investigates the ways in which technology might support 
active learning could be promoted (Freeman et. al. 2014). 


For AP Calculus and AP Physics students, the prevalence of instructional videos, 
simulations, and applets provides a relatively new resource to turn to when students are 
uncertain about concepts and skills. Most of these resources are available at no cost to 
students (and these same resources are available to students outside the United States). 
This study offers a brief glimpse into students’ self-directed use of supplementary 
instructional resources: recall, four-fifths of AP Calculus and AP Physics students reported 
using the internet-based instructional resources. At present, this is an underreported area of 
research, both in the extent of student use of these resources as a supplement to face-to- 
face instruction in advanced mathematics and science, and in their relative impact on 
student learning and engagement. To fully understand the affordances and constraints of 
these resources, a more robust research agenda should be articulated and supported to 
understand how students use technology on their own to enhance their learning and 
achievement in mathematics and science. In addition, future large scale assessments could 
include questionnaires with a more comprehensive set of prompts specifically developed to 
focus on the goals, use, and challenges of instructional technology. 


Teachers and Teaching 


The calculus and physics teachers in this study share similar characteristics. They are older, 
more experienced, and most have earned a master’s degree. Approximately 70% of 
calculus students and 50% of physics students had teachers who had been teaching before 


78 Oo CollegeBoard 


2000, with about one-third of AP and non-AP teachers nearing retirement age. The relative 
seniority of teachers, combined with recent flat or declining enrollments in secondary math 
and science programs, and higher turnover rates for novice teachers, should raise concerns 
for the future preparation of students who are pursuing STEM intensive college and career 
pathways. In addition, secondary mathematics and science tends to have higher turnover 
rates than their counterparts (Ingersoll et.al. 2014). These issues related to the teacher 
education pipeline for mathematics and science education have been known for years. 
Furthermore, Ingersoll, Merrill, and Stuckey (2014) also noted that 


... while these trends raise important questions, until recently we have seen 
little awareness or discussion of them or their implications—whether by 
researchers, by policymakers, by educators, or by the public ... if these 
trends do indeed continue, there will be large implications, with serious 
financial, structural, and educational consequences for America’s educational 
system (p. 27). 


Well prepared, highly qualified teachers are critical to ensuring students’ opportunity to 
learn, and their pursuit of STEM majors and careers. So, we reiterate the need for 
policymakers, and the public, to give attention to the need to improve the recruitment and 
retention of secondary mathematics and science teachers. As noted in section 5, advanced 
math and physics students have limited interest in education, which suggests that the 
teaching profession has an image problem among the next generation of math and science 
majors. This negative image is often reinforced by faculty, peers, and media when 
mathematics and science majors are encouraged to pursue more lucrative or prestigious 
options. However, recruitment is only part of the puzzle. 


Some of the top reasons teachers give for leaving the profession include working conditions, 
lack of opportunity for professional growth, and input into decision making. Most of the 
teachers in this study had extensive teaching experience and most had completed a 
master’s degree. Yet, finding ways to assist struggling students is reported to be a challenge 
by these teachers, even with students who have a strong interest in these courses. 
Likewise, less than one-fifth of students enrolled in physics had teachers who described 
themselves as very highly confident using inquiry methods. In almost any other industry or 
profession, such difficulty hiring and retaining enough qualified employees to meet the 
demands of the market would bring about a swift response; the political will to improve 
recruitment, retention, and work conditions would be strong. Leadership needs to emerge to 
communicate and implement a vision of attractive schools—and to enact policies that 
support effective induction experiences for candidates eager to develop professional 
knowledge and instructional practice. 
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